Papers by Liyuan Liu
Understanding the Difficulty of Training Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Admin (Adaptive model initialization) is more stable, converges faster, and leads to better performance. |
| Approach: | They propose a model initialization algorithm to stabilize early training and unleash its full potential in the late stage. |
| Outcome: | The proposed model initialization method stabilizes early training and unleashes full potential in late stage. |
Facet-Aware Evaluation for Extractive Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | lexical overlap is a common evaluation metric for extractive summarization, but recent studies reveal its limitations. |
| Approach: | They propose a facet-aware evaluation setup for better assessment of information coverage in extractive summaries. |
| Outcome: | The proposed evaluation setup improves human correlation with extractive summarization datasets and improves comparative analysis. |
Efficient Contextualized Representation: Language Model Pruning for Sequence Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing efforts to train pre-trained language models have brought significant improvements to various NLP applications. |
| Approach: | They propose to compress bulky LMs while preserving useful information for a specific task. |
| Outcome: | The proposed method can detach any layer without affecting others, and stretch shallow and wide LMs to be deep and narrow. |
ProBench: Judging Multimodal Foundation Models on Open-ended Multi-domain Expert Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Solving expert-level multimodal tasks requires strong user query understanding, domain-specific knowledge, and advanced reasoning abilities. |
| Approach: | They propose a benchmark of open-ended user queries encapsulating professional expertise and advanced reasoning. |
| Outcome: | The proposed benchmark is publicly accessible at TBC. |
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure (2026.acl-demo)
Copied to clipboard
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, Zhuohan Xie
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
Looking Beyond Label Noise: Shifted Label Distribution Matters in Distantly Supervised Relation Extraction (D19-1)
Copied to clipboard
| Challenge: | Existing studies on DS-based relation extraction (RE) methods focus on handling label noise, but other factors may have been overlooked. |
| Approach: | They propose a method to automatically adjust DS-RE models to a shifted label distribution problem . they find this problem exists in real-world DS datasets and can be overcome . |
| Outcome: | The proposed method achieves consistent performance gains on DS-trained models with an up to 23% relative F1 improvement, which verifies their assumptions. |
On the Transformer Growth for Progressive BERT Training (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods only conduct network growth in a single dimension, but compound growth operators are beneficial for multiple dimensions. |
| Approach: | They propose a method to train BERT progressively using a Transformer model and explore alternative growth operators in each dimension via controlled comparison. |
| Outcome: | The proposed method speeds up BERT pre-training by 73.6% and 82.2% for the base and large models respectively while achieving comparable performances. |
CrossWeigh: Training Named Entity Tagger from Imperfect Annotations (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) models can identify labels in 5.38% of test sentences . a framework to handle label mistakes during NER model training is proposed . |
| Approach: | They propose a framework to manually correct label mistakes in named entity recognition (NER) they aim to improve the accuracy of models by re-evaluating popular models on corrected test sets . |
| Outcome: | The proposed framework can detect label mistakes in 5.38% of test sentences . the proposed framework improves on three datasets with a high-performance model . |
Reliability-aware Dynamic Feature Composition for Name Tagging (P19-1)
Copied to clipboard
| Challenge: | Word embeddings are used to encode semantic information, but their quality is not consistent across the vocabulary due to the long-tail distribution of word frequency. |
| Approach: | They propose a reliability-aware name tagging model that uses word frequency to indicate word quality . they propose to use word frequency-based reliability signals to dynamically select and compose features . |
| Outcome: | The proposed model outperforms the baseline model on OntoNotes 5.0 and up to 5% gain on cross-genre data sets. |
Learning Named Entity Tagger using Domain-Specific Dictionary (D18-1)
Copied to clipboard
| Challenge: | Existing methods to build reliable named entity recognition systems require large amounts of manually-labeled training data. |
| Approach: | They propose a revised fuzzy CRF layer to handle tokens with multiple possible labels to address noisy distant supervision. |
| Outcome: | The proposed model can handle tokens with multiple possible labels under the traditional framework and improves on the existing model with a new Tie or Break scheme. |
Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for Sequence Labeling require high-quality annotations, but imperfect annotations are relatively easy to obtain from crowdsourcing (noisy labels) Existing approaches to learn a model without knowing the underlying ground truth label sequences in the target domain are expensive and time-consuming. |
| Approach: | They propose a framework Consensus Network that can be trained on annotations from multiple sources. |
| Outcome: | The proposed framework improves on learning with crowd annotations and unsupervised cross-domain model adaptation in two practical settings. |