Papers by Leyang Cui
Knowledge Verification to Nip Hallucination in the Bud (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models generate responses that sound plausible but contradict factual knowledge, a phenomenon known as hallucination. |
| Approach: | They propose a novel approach to align large language models to evaluate knowledge boundaries based on external knowledge to reduce hallucinations . |
| Outcome: | The proposed approach reduces hallucinations across six benchmarks using foundation LLMs of varying backbones and scales. |
Enhancing Grammatical Error Correction Systems with Explanations (2023.acl-long)
Copied to clipboard
| Challenge: | To help language learners better understand why the GEC system makes a correction, the causes of errors and the corresponding error types are two key factors. |
| Approach: | They propose to annotate large dataset with evidence words and grammatical error types to help language learners better understand corrections. |
| Outcome: | The proposed model can be validated by human evaluation and can be used to help second-language learners decide whether to accept a correction suggestion and understand the associated grammar rule. |
Multi-Granularity Optimization for Non-Autoregressive Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive machine translation suffers severe performance deterioration due to the naive independence assumption. |
| Approach: | They propose a method which collects model behaviours on translation segments of various granularities and integrates feedback for backpropagation to reduce latency. |
| Outcome: | Experiments on four benchmark datasets show that the proposed method outperforms baseline models trained with cross-entropy loss and achieves the best performance on WMT’16 EnRo and highly competitive results on WTM’14 EnDe. |
Thinking Out Loud: Do Reasoning Models Know When They’re Right? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large reasoning models (LRMs) have recently demonstrated impressive capabilities in complex reasoning tasks by leveraging increased test-time computation and exhibiting behaviors reminiscent of human-like self-reflection. |
| Approach: | They analyze verbalized confidence, how models articulate their certainty, as a lens into the nature of self-reflection in large reasoning models. |
| Outcome: | The proposed model exhibits human-like self-reflection in reasoning tasks, but how this ability interacts with other model behaviors remains underexplored. |
Investigating Non-local Features for Neural Constituency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Constituency parsers have been able to achieve competitive performance by using local features. |
| Approach: | They propose to inject non-local features into the training process of a local span-based parser by predicting constituent n-gram non-local patterns and ensuring consistency between constituents and local constituents. |
| Outcome: | The proposed method outperforms the self-attentive parser in multi-lingual and zero-shot cross-domain settings. |
Knowledge Enhanced Fine-Tuning for Better Handling Unseen Entities in Dialogue Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for dialogue generation use an external knowledge base to generate appropriate responses. |
| Approach: | They propose to use an external knowledge base to generate appropriate responses for unseen entities. |
| Outcome: | Experiments on two dialogue corpus show that pre-trained models perform poorly with unseen entities. |
Lost in Literalism: How Supervised Training Shapes Translationese in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit translationese errors and generate unexpected unnatural translations . Neural machine translation (NMT) has become the dominant method in machine translation research . |
| Approach: | They evaluate the prevalence of translationese in LLM-generated translations and investigate its roots during supervised fine-tuning. |
| Outcome: | The proposed methods reduce translationese while improving translation naturalness . the proposed methods are validated by human evaluations and automatic metrics . |
Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal (2024.acl-long)
Copied to clipboard
Jianheng Huang, Leyang Cui, Ante Wang, Chengyi Yang, Xinting Liao, Linfeng Song, Junfeng Yao, Jinsong Su
| Challenge: | Existing methods to train LLMs on previous training data are not feasible in real-world applications because of catastrophic forgetting. |
| Approach: | They propose a framework that uses the LLM to generate synthetic instances for rehearsal and refine the instance outputs based on the synthetic inputs. |
| Outcome: | The proposed framework achieves superior or comparable performance compared to conventional rehearsal-based approaches while being more data-efficient. |
MuTual: A Dataset for Multi-Turn Dialogue Reasoning (2020.acl-main)
Copied to clipboard
| Challenge: | Existing non-task oriented dialogue systems can yield a relevant and fluent response, but sometimes make logical mistakes because of weak reasoning capabilities. |
| Approach: | They propose a dataset for multi-turn dialogue reasoning that uses annotated dialogues to train a machine to handle various reasoning problems. |
| Outcome: | Empirical results show that state-of-the-art methods only reach 71%, far behind human performance of 94%. |
On Commonsense Cues in BERT for Solving Commonsense Tasks (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models can capture syntactic features, semantic information and factual knowledge, but structured commonsense knowledge is not captured well. |
| Approach: | They quantitatively investigate the presence of structural commonsense cues in BERT when solving commonsensense tasks and the importance of such cue for the model prediction. |
| Outcome: | The presence of commonsense knowledge is positively correlated to the model accuracy. |
Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work uses large language models (LLMs) to evaluate natural language process tasks, but there are shortcomings in current LLMs. |
| Approach: | They examine the alignment between LLM evaluators and human annotators by comparing conventional and alignment tasks with different evaluation criteria. |
| Outcome: | The proposed models excel in general criteria, such as fluency, but face challenges with complex criteria, including numerical reasoning. |
Template-Based Named Entity Recognition Using BART (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fewshot NER do not make full use of knowledge transfer in NER model parameters. |
| Approach: | They propose a template-based method for NER that treats NER as a language model ranking problem in a sequence-to-sequence framework. |
| Outcome: | The proposed method achieves 92.55% F1 score on the CoNLL03 task and significantly better than fine-tuning BERT 10.88%, 15.34%, and 11.73% F1 scores on the MIT Movie, the ATIS, and the MATLAB task. |
Spotting AI’s Touch: Identifying LLM-Paraphrased Spans in Text (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing work focuses on detecting (partially) AI-generated texts, but paraphrasing is commonly employed in various application scenarios for text refinement and diversity. |
| Approach: | They propose a framework for paraphrased text span detection that takes in the full text and assigns each sentence with a score indicating the paraphrasing degree. |
| Outcome: | The proposed framework can detect paraphrased text spans within a text . it takes in the full text and assigns each sentence with a score indicating the paraphrasing degree. |
GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated impressive performance across various mathematical reasoning benchmarks. |
| Approach: | They introduce an adversarial grade school math dataset and explore whether LLMs can be more robust when questions are slightly changed. |
| Outcome: | The proposed method generates and verifies each intermediate thought based on its reasoning goal and calculation result. |
Benchmarking and Improving Long-Text Translation with Large Language Models (2024.findings-acl)
Copied to clipboard
Longyue Wang, Zefeng Du, Wenxiang Jiao, Chenyang Lyu, Jianhui Pang, Leyang Cui, Kaiqiang Song, Derek Wong, Shuming Shi, Zhaopeng Tu
| Challenge: | Recent studies have illuminated the promising capabilities of large language models (LLMs) in handling long texts. |
| Approach: | They construct a benchmark dataset specifically designed for the finetuning and evaluation of large language models (LLMs) they compare LLMs with MT models and find they exhibit shortcomings in long-text domains . |
| Outcome: | The proposed model performs better in long-text translation, and its performance diminishes as document size increases. |
Cross-domain Generalization for AMR Parsing (2022.emnlp-main)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) parsing aims to predict an AMR graph from textual input. |
| Approach: | They evaluate five representative AMR parsers on five domains and analyze challenges to cross-domain parsing. |
| Outcome: | The proposed method reduces the domain distribution divergence of text and AMR features on two out-of-domain sets. |
RobustGEC: Robust Grammatical Error Correction Against Subtle Context Perturbation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Grammatical Error Correction (GEC) systems perform well in academic benchmarks, but in practical applications they may not correct errors when users perform irrelevant modifications. |
| Approach: | They propose a benchmark to evaluate the context robustness of Grammatical Error Correction systems. |
| Outcome: | The proposed method improves the accuracy of errors corrected by human annotations. |
Non-autoregressive Text Editing with Copy-aware Latent Alignments (2023.emnlp-main)
Copied to clipboard
| Challenge: | Seq2Edit approaches still face several challenges such as inflexibility in generation and difficulty in generalizing to other languages. |
| Approach: | They propose a non-autoregressive text editing method that models the edit process with latent CTC alignments and introduces the copy operation into the edit space. |
| Outcome: | The proposed method outperforms existing Seq2Edit models and achieves similar or even better results than Seq1Edit with over 4 speedup. |
FactMix: Using a Few Labeled In-domain Examples to Generalize to Cross-domain Named Entity Recognition (2022.coling-1)
Copied to clipboard
| Challenge: | Existing approaches for few-shot Named Entity Recognition (NER) are evaluated mainly under in-domain settings, but little is known about how these models perform in cross-domain NER using labeled in- domain examples. |
| Approach: | They propose to use a rationale-centric data augmentation method to improve model generalization ability by allowing model to learn from a few labeled examples in a new target domain. |
| Outcome: | The proposed method improves the performance of cross-domain NER tasks compared to the counterfactual data augmentation and prompt-tuning methods. |
Challenges to Open-Domain Constituency Parsing (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing findings on cross-domain constituency parsing are only made on a limited number of domains. |
| Approach: | They manually annotate a high-quality constituency treebank containing five domains and analyze challenges to open-domain constituency parsing using a set of linguistic features. |
| Outcome: | The proposed model significantly improves the performance of the proposed model on the domain-variant features. |
Effidit: An Assistant for Improving Writing Efficiency (2023.acl-demo)
Copied to clipboard
Shuming Shi, Enbo Zhao, Wei Bi, Deng Cai, Leyang Cui, Xinting Huang, Haiyun Jiang, Duyu Tang, Kaiqiang Song, Longyue Wang, Chenyan Huang, Guoping Huang, Yan Wang, Piji Li
| Challenge: | Effidit is a digital writing assistant that provides three modules to help users write faster and more efficiently. |
| Approach: | They present Effidit, a digital writing assistant that provides three modules to help users write higher-quality text more efficiently. |
| Outcome: | Effidit expands the capabilities of a typical writing assistant by providing three modules . Effit can help users create their own text faster and more efficiently . |
Does Chinese BERT Encode Word Structure? (2020.coling-main)
Copied to clipboard
| Challenge: | Existing work has focused on analyzing the features captured by representative models such as BERT . however, little work has investigated word features for character languages such as Chinese . |
| Approach: | They investigate Chinese BERT using attention weight distribution statistics and probing tasks to understand word features. |
| Outcome: | The proposed model improves syntactic, semantic and word sense knowledge on a wide range of NLP tasks. |
Alleviating Hallucinations of Large Language Models through Induced Hallucinations (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that large language models generate inaccurate or fabricated information, a phenomenon known as hallucinations. |
| Approach: | They propose a simple strategy to induce-then-contrast decode LLMs to enhance their factuality . they first induce hallucinations from the original model and penalize them . |
| Outcome: | The proposed strategy improves factuality of large language models across task formats, model sizes, and model families. |
Explicit Syntactic Guidance for Neural Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing text generation models follow the sequence-to-sequence paradigm . generative grammar suggests humans generate language by learning language grammar . |
| Approach: | They propose a syntax-guided generation schema that searches the syntax tree in a top-down direction. |
| Outcome: | The proposed method outperforms autoregressive baselines on paraphrase generation and machine translation. |
MAGE: Machine-generated Text Detection in the Wild (2024.acl-long)
Copied to clipboard
Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, Yue Zhang
| Challenge: | Existing research has focused on evaluating detection methods for specific domains or language models. |
| Approach: | They build a testbed to detect texts from diverse human writings and LLMs using different detection methods. |
| Outcome: | Empirical results show that the top performing detector can identify 84.12% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios. |
Hierarchically-Refined Label Attention Network for Sequence Labeling (D19-1)
Copied to clipboard
| Challenge: | Conditional random fields (CRF) is a powerful model for statistical sequence labeling, but it does not give much information gain over strong neural encoding. |
| Approach: | They propose a hierarchically-refined label attention network which captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. |
| Outcome: | The proposed model improves POS tagging accuracy and speeds up training and testing compared to the current model. |
What Have We Achieved on Text Summarization? (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text summarization have been investigated, but there are still gaps between them and human professionals. |
| Approach: | They analyze 8 major sources of errors on 10 representative summarization models manually. |
| Outcome: | Aiming to gain more understanding of summarization systems with respect to their strengths and limitations on a fine-grained syntactic and semantic level, we use 8 major sources of errors on 10 representative summarizing models. |
Solving Aspect Category Sentiment Analysis as a Text Generation Task (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for Aspect category sentiment analysis use pre-trained language models to learn aspect category-specific representations. |
| Approach: | They propose to make use of pre-trained language models by casting the ACSA tasks into natural language generation tasks, using natural language sentences to represent the output. |
| Outcome: | The proposed method gives the best reported results, having large advantages in few-shot and zero-shot settings. |
Selection-p: Self-Supervised Task-Agnostic Prompt Compression for Faithfulness and Transferability (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities when leveraging in-context learning. |
| Approach: | They propose a method that discretizes uninformative tokens using a self-supervised pre-training technique. |
| Outcome: | The proposed method achieves state-of-the-art performance across classification tasks while requiring only 0.8% decrease in performance. |
LogiCoT: Logical Chain-of-Thought Instruction Tuning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on self-instruction tuning has focused on enhancing the general proficiency of models. |
| Approach: | They propose a new instruction-tuning dataset for Logical Chain-of-Thought reasoning with GPT-4 that harvests instructions for prompting GPT to generate chain-of thought rationales. |
| Outcome: | The proposed dataset enables the model to generate chain-of-thought rationales with GPT-4. |
Uni-Encoder: A Fast and Accurate Response Selection Paradigm for Generation-Based Dialogue Systems (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to decode a given context-candidate pair are expensive and time-consuming. |
| Approach: | They propose a new paradigm that keeps full attention over each pair while only encoding the context once. |
| Outcome: | The proposed paradigm achieves new state-of-the-art results on four benchmark datasets with high computational efficiency. |
Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs (2025.findings-naacl)
Copied to clipboard
| Challenge: | a new framework for complex reasoning with LLMs is developed to improve reasoning proof accuracy and interpretability. |
| Approach: | They propose to use LLMs to generate search logs that can be interpreted into human-readable reasoning proofs. |
| Outcome: | The proposed framework improves reasoning accuracy but lacks interpretability due to black-box nature of the solvers. |
EDeR: Towards Understanding Dependency Relations Between Events (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on event relation extraction focuses on hierarchical, temporal and causal relations but ignores the interdependence between events. |
| Approach: | They propose to use a human-annotated Event Dependency Relation dataset to identify event dependency relations between two events. |
| Outcome: | The proposed dataset integrates existing annotations with the OntoNotes dataset and shows that recognizing such event dependency relations can further benefit critical NLP tasks, including semantic role labelling and co-reference resolution. |
Not All Preference Pairs Are Created Equal: A Recipe for Annotation-Efficient Iterative Preference Learning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown remarkable capabilities to understand and generate human languages, supporting applications such as question answering, coding, and psychological counseling. |
| Approach: | They propose strategies to save annotation budgets while achieving competitive or even better performances for iterative preference learning. |
| Outcome: | The proposed methods save annotation budgets while achieving better performance. |
Making the Best Use of Review Summary for Sentiment Analysis (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for sentiment analysis of user reviews are limited to a few examples. |
| Approach: | They propose a hierarchically-refined attention model that exploits the sentimental distribution of a review and its corresponding summary. |
| Outcome: | The proposed model can make better use of user-written summaries for review sentiment analysis and is more effective compared to existing methods when the user summary is replaced with summary generated by an automatic summarization system. |