Papers by Fei Sun

Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts (2024.findings-acl)

Challenge:	Existing methods for creating versatile MLLMs rely on joint training with paired instruction data, which is resource-intensive and challenging to extend to new modalities.
Approach:	They propose a new paradigm for multimodal large language models by reusing modality encoders and merging LLM parameters.
Outcome:	The proposed model retains the modal understanding capabilities of each original model.

Copied to clipboard

Ganesh Jawahar, Haichuan Yang, Yunyang Xiong, Zechun Liu, Dilin Wang, Fei Sun, Meng Li, Aasish Pappu, Barlas Oguz, Muhammad Abdul-Mageed, Laks Lakshmanan, Raghuraman Krishnamoorthi, Vikas Chandra

Stylized Knowledge-Grounded Dialogue Generation via Disentangled Template Rewriting (2022.naacl-main)

Challenge:	Neural architecture search (NAS) uses weight-sharing supernets to generate diverse subnetworks without retraining.
Approach:	They propose a weight-sharing supernet that leverages mixture-of-experts to enhance supernet model expressiveness with minimal training overhead.
Outcome:	The proposed method achieves state-of-the-art (SoTA) performance in NAS for fast machine translation models, surpassing NAS-BERT and AutoDistil across various model sizes.

Copied to clipboard

Qingfeng Sun, Can Xu, Huang Hu, Yujing Wang, Jian Miao, Xiubo Geng, Yining Chen, Fei Xu, Daxin Jiang

Order Doesn’t Matter, But Reasoning Does: Training LLMs with Order-Centric Augmentation (2025.emnlp-main)

Challenge:	Existing knowledge-grounded dialogue generation models only produce pedantic responses, which lacks emotion and attraction compared with the responses with polite style, positive and negative sentiments.
Approach:	They propose a method which generates responses via combing disentangled style templates and content templates.
Outcome:	The proposed method improves on evaluation metrics compared with state-of-the-art methods.

Copied to clipboard

Qianxi He, Qianyu He, Jiaqing Liang, Weikang Zhou, Zeye Sun, Fei Yu, Yanghua Xiao

Dependency Parsing as MRC-based Span-Span Prediction (2022.acl-long)

Challenge:	Logical reasoning is essential for large language models (LLMs) to ensure accurate and coherent inferences.
Approach:	They propose an order-centric data augmentation framework based on commutativity in logical reasoning that randomly shuffles independent premises to introduce condition order augmentation.
Outcome:	The proposed framework improves LLMs’ reasoning performance and adaptability to diverse logical structures.

Copied to clipboard

Leilei Gan, Yuxian Meng, Kun Kuang, Xiaofei Sun, Chun Fan, Fei Wu, Jiwei Li

Syntactic Text Processing

Challenge:	Existing methods for dependency parsing address the issue that edges should be constructed at the text span/subtree level rather than word level.
Approach:	They propose a method that constructs dependency trees by directly modeling span-span relations by modeling subtree-subtree relationships.
Outcome:	The proposed method constructs dependency trees by modeling span-span relations . it can retrieve missing spans in the span proposal stage, which leads to higher recall .

SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling (2025.emnlp-main)

Copied to clipboard

Krishna C Puvvada, Faisal Ladhak, Santiago Akle Serano, Cheng-Ping Hsieh, Shantanu Acharya, Somshubra Majumdar, Fei Jia, Samuel Kriman, Simeng Sun, Dima Rekesh, Boris Ginsburg

The Mirage of Model Editing: Revisiting Evaluation in the Wild (2025.acl-long)

Challenge:	Existing decoder-only models struggle with context lengths beyond their training distribution.
Approach:	They propose a causal Transformer architecture that generalizes robustly to sequence lengths longer than seen during training.
Outcome:	The proposed decoder-only architecture can generalize robustly to longer contexts . it is more computationally efficient than the standard Transformer architecture, the authors say .

Copied to clipboard

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Qi Cao, Dawei Yin, Huawei Shen, Xueqi Cheng

Question Answering

Challenge:	despite near-perfect results, effectiveness of model editing in real-world applications remains unclear.
Approach:	They propose QAEdit and WILD to better reflect real-world use of model editing . they propose a benchmark aligned with widely used question answering datasets and a task-agnostic evaluation framework .
Outcome:	The proposed QAEdit benchmark and WILD evaluation framework show that current models perform worse than previously reported.

Evolving Knowledge Distillation with Large Language Models and Active Learning (2024.lrec-main)

Copied to clipboard

Chengyuan Liu, Fubang Zhao, Kun Kuang, Yangyang Kang, Zhuoren Jiang, Changlong Sun, Fei Wu

Knowledge Stimulated Contrastive Prompting for Low-Resource Stance Detection (2022.findings-emnlp)

Challenge:	Existing studies have focused on the direct use of large language models for text generation and labeling, without fully exploring their potential to comprehend the target task and acquire valuable knowledge.
Approach:	They propose to distill the knowledge of large language models into smaller models by generating annotated data.
Outcome:	The proposed method improves the performance of small domain models while enhancing the ability of large language models.

Copied to clipboard

Kai Zheng, Qingfeng Sun, Yaming Yang, Fei Xu

DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modeling (2025.emnlp-main)

Challenge:	Stance Detection Tasks require background knowledge especially when there is no explicit target mentioned in text.
Approach:	They propose a masked language prompt joint contrastive learning approach to stimulate the knowledge inherit from pre-trained models.
Outcome:	The proposed model is effective in stance detection on three benchmarks.

Copied to clipboard

Hao Sun, Zile Qiao, Bo Wang, Guoxin Chen, Yingyan Hou, Yong Jiang, Pengjun Xie, Fei Huang, Yan Zhang

De-Biased Court’s View Generation with Causality (2020.emnlp-main)

Challenge:	Retrieval-Augmented Generation (RAG) systems have emerged as a pivotal methodology for enhancing Large Language Models (LLMs).
Approach:	They propose a framework that decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding.
Outcome:	The proposed framework decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding.

Copied to clipboard

Yiquan Wu, Kun Kuang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Jun Xiao, Yueting Zhuang, Luo Si, Fei Wu

Improving Multi-turn Dialogue Modelling with Utterance ReWriter (P19-1)

Challenge:	Existing approaches to court’s view generation can be used to address this problem, but neglecting the confounding bias in data can limit the model performance and pollute learning outcomes.
Approach:	They propose a novel Attentional and Counterfactual based Natural Language Generation method consisting of an attentional encoder and a pair of innovative counterfactual decoders to generate judgment-discriminative court's views.
Outcome:	The proposed method is able to generate judgment-discriminative court's views (both supportive and non-supportive views) under both quantitative and qualitative evaluation metrics.

Copied to clipboard

Hui Su, Xiaoyu Shen, Rongzhi Zhang, Fei Sun, Pengwei Hu, Cheng Niu, Jie Zhou

Paraphrasing

Challenge:	Recent research has achieved impressive results in single-turn dialogue modelling, but multi-turn models still remain challenging.
Approach:	They propose to rewrite human utterances as a pre-process to help multi-turn dialgoue modelling.
Outcome:	The proposed architecture achieves remarkably good performance on the utterance rewriting task.

Browse and Concentrate: Comprehending Multimodal Content via Prior-LLM Context Fusion (2024.acl-long)

Copied to clipboard

Ziyue Wang, Chi Chen, Yiqi Zhu, Fuwen Luo, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Maosong Sun, Yang Liu

Multimodality

Challenge:	Multimodal Large Language Models (MLLMs) lack understanding of multi-image and interleaved inputs due to the visual features encoded by frozen encoders before being fed into the LLM backbone.
Approach:	They propose a two phase paradigm to enable in-depth multimodal context fusion prior to feeding the features into LLMs.
Outcome:	The proposed paradigm boosts the performance on 7 multi-image scenarios, contributing to increments on average accuracy by 2.13% and 7.60% against strong MLLMs baselines with 3B and 11B LLMs, respectively.

Doc2Bot: Accessing Heterogeneous Documents via Conversational Bots (2022.findings-emnlp)

Copied to clipboard

Haomin Fu, Yeqin Zhang, Haiyang Yu, Jian Sun, Fei Huang, Luo Si, Yongbin Li, Cam Tu Nguyen

Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs (2024.emnlp-main)

Challenge:	Documents contain various structures that hinder the ability of machines to comprehend . user information needs are often underspecified, and the nature of heterogeneous documents poses challenges.
Approach:	They propose a dataset for building machines that help users seek information via conversations . their dataset contains over 100,000 turns based on Chinese documents from five domains .
Outcome:	The proposed tasks are challenging and worthy of further research.

Copied to clipboard

Chengyuan Liu, Shihang Wang, Lizhi Qing, Kun Kuang, Yangyang Kang, Changlong Sun, Fei Wu

Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialogue State Tracking (2021.acl-short)

Challenge:	Large Language Models (LLMs) struggle when it comes to specialized domains due to limited domain-specific knowledge.
Approach:	They propose an adaptive method that automatically identifies valuable words from a given domain vocabulary.
Outcome:	The proposed method has been validated on three Chinese datasets and performed on general tasks.

Copied to clipboard

Yinpei Dai, Hangyu Li, Yongbin Li, Jian Sun, Fei Huang, Luo Si, Xiaodan Zhu

Table-based Fact Verification With Salience-aware Learning (2021.findings-emnlp)

Challenge:	Existing dialog state tracking models neglect rich structural information in a dataset.
Approach:	They propose to use curriculum learning to leverage dialog state tracking data . they propose a model-agnostic framework that pre-trains a DST model with schema information .
Outcome:	The proposed framework improves performance over a transformer-based and RNN-based model on WOZ2.0 and MultiWOZ2.1.

Copied to clipboard

Fei Wang, Kexuan Sun, Jay Pujara, Pedro Szekely, Muhao Chen

Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation (2021.findings-emnlp)

Challenge:	Existing methods for fact verification use tabular data with tokens, but training requires labeled training data.
Approach:	They propose a system that identifies token-level salience in the statement with probing-based saliency estimation.
Outcome:	The proposed system improves on TabFact benchmark by replacing non-salient terms with tokens.

Copied to clipboard

Yuexiang Xie, Fei Sun, Yang Deng, Yaliang Li, Bolin Ding

Turn Waste into Worth: Rectifying Top-k Router of MoE (2024.emnlp-main)

Challenge:	Existing methods to evaluate factual consistency in text summarization neglect the intrinsic cause of factual inconsistency or rely on auxiliary tasks.
Approach:	They propose a method to evaluate the factual consistency in text summarization via counterfactual estimation, which formulates the causal relationship between source document, generated summary, and the language prior.
Outcome:	The proposed metric improves correlation with human judgments and convenience of usage on three public abstractive text summarization datasets.

Copied to clipboard

Zhiyuan Zeng, Qipeng Guo, Zhaoye Fei, Zhangyue Yin, Yunhua Zhou, Linyang Li, Tianxiang Sun, Hang Yan, Dahua Lin, Xipeng Qiu

Green & Sustainable NLP

Challenge:	Top-k router suffers from redundancy computation and memory costs due to unbalanced routing . some experts are overflow, where exceeding tokens are dropped, while others are empty, which are padded with zeros, negatively impacting model performance.
Approach:	They propose a top-k router that is unbalanced and uses a multi-gPU system to handle dropped tokens and padding.
Outcome:	The proposed model surpasses the top-1 router by 4.7% in terms of performance . the top-k router suffers from redundancy computation and memory costs .

ConRPG: Paraphrase Generation using Contexts as Regularizer (2021.emnlp-main)

Copied to clipboard

Yuxian Meng, Xiang Ao, Qing He, Xiaofei Sun, Qinghong Han, Fei Wu, Chun Fan, Jiwei Li

RMTBench: Benchmarking LLMs Through Multi-Turn User-Centric Role-Playing (2025.findings-emnlp)

Challenge:	Existing methods for paraphrase generation lack reliable supervision signals.
Approach:	They propose an unsupervised paradigm for paraphrase generation based on contextual language models, candidate filtering and paraphrase model training based upon the selected candidates.
Outcome:	The proposed paradigm outperforms existing paraphrase generation methods in supervised and unsupervised setups.

Copied to clipboard

Hao Xiang, Tianyi Tang, Yang Su, Bowen Yu, An Yang, Fei Huang, Yichang Zhang, Yaojie Lu, Hongyu Lin, Xianpei Han, Jingren Zhou, Junyang Lin, Le Sun

Modeling Complex Dialogue Mappings via Sentence Semantic Segmentation Guided Conditional Variational Auto-Encoder (2022.findings-emnlp)

Challenge:	Existing benchmarks focus on character-centric approach and fail to reflect real-world applications.
Approach:	RMTBench is a user-centric bilingual role-playing benchmark featuring 80 diverse characters and over 8,000 dialogue rounds.
Outcome:	RMTBench features 80 diverse characters and over 8,000 dialogue rounds.

Copied to clipboard

Bin Sun, Shaoxiong Feng, Yiwei Li, Weichao Wang, Fei Mi, Yitong Li, Kan Li

Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation (2022.findings-emnlp)

Challenge:	Existing efforts to identify and avoid CDM to facilitate dialogue learning failed to solve the problem.
Approach:	They propose a Sentence Semantic Segmentation guided Conditional Variational Auto-Encoder which can model and take advantage of the CDM data.
Outcome:	The proposed method can model and take advantages of the CDM data.

Copied to clipboard

Zhexin Zhang, Jiale Cheng, Hao Sun, Jiawen Deng, Fei Mi, Yasheng Wang, Lifeng Shang, Minlie Huang

Towards Interactivity and Interpretability: A Rationale-based Legal Judgment Prediction Framework (2022.emnlp-main)

Challenge:	Existing methods to detect toxic generation of pretrained language models rely on templates, data extraction, crowdsourcing workers or automatic generation.
Approach:	They propose a method to construct adversarial contexts conditioned on a given response . they augment existing dataset BAD+ and construct a new dataset B AD+ .
Outcome:	The proposed method can detect toxic or biased content in large pretrained language models.

Copied to clipboard

Yiquan Wu, Yifei Liu, Weiming Lu, Yating Zhang, Jun Feng, Changlong Sun, Fei Wu, Kun Kuang

Rapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese (2024.lrec-main)

Challenge:	Existing models lack interpretability due to the neglect of rationale in the prediction process.
Approach:	They propose a rationale-based legal judgment prediction framework that follows the judge's real trial logic and provides good interactivity and interpretability.
Outcome:	The proposed framework provides good interactivity and interpretability which enables practical use.

Copied to clipboard

Yikun Sun, Zhen Wan, Nobuhiro Ueda, Sakiko Yahata, Fei Cheng, Chenhui Chu, Sadao Kurohashi

PRDetect: Perturbation-Robust LLM-generated Text Detection Based on Syntax Tree (2025.findings-naacl)

Challenge:	Recent advances in large language models (LLMs) have aimed to refine their capacity to accurately follow human instructions and navigate intricate scenarios.
Approach:	They propose a method that uses a set of instructions to translate English into Japanese and then generates Japanese instruction data using GPT-4.
Outcome:	The proposed method outperforms Japanese-Alpaca models in the evaluation benchmarks without human references.

Copied to clipboard

Xiang Li, Zhiyi Yin, Hexiang Tan, Shaoling Jing, Du Su, Yi Cheng, Huawei Shen, Fei Sun

Syntactic Text Processing

Challenge:	Recent methods for detecting LLM-generated text have shown impressive performance, but in real-world scenarios, users often introduce perturbations to the text.
Approach:	They propose a method that detects syntactic trees that are minimally affected by perturbations and exhibit distinct differences between human-written and LLM-generated text.
Outcome:	The proposed method shows that it is significantly better against perturbations on the HC3 and GPT-3.5-mixed datasets and also has the shortest time expenditure.

KS-Lottery: Finding Certified Lottery Tickets for Multilingual Transfer in Large Language Models (2025.naacl-long)

Copied to clipboard

Fei Yuan, Chang Ma, Shuai Yuan, Qiushi Sun, Lei Li

Detecting Stealthy Backdoor Samples based on Intra-class Distance for Large Language Models (2025.findings-emnlp)

Challenge:	Existing studies have shown that a small subset of parameters is highly effective in fine-tuning . prior work shows that there are a few additional parameters corresponding to an intrinsic dimension in a well-trained Large Language Model.
Approach:	They propose a method to identify a small subset of LLM parameters highly effective in multilingual fine-tuning.
Outcome:	The proposed method can find the certified winning tickets in the embedding layer, and fine-tuning on the found parameters is guaranteed to perform as well as full fine- tuning.

Copied to clipboard

Jinwen Chen, Hainan Zhang, Fei Sun, Qinnan Zhang, Sijia Wen, Ziwei Wang, Zhiming Zheng

Robustness in NLP

Challenge:	Existing detectors use classifier-style probability signals or rely on rewriting, which can degrade quality and introduce new triggers.
Approach:	They propose to efficiently remove poisoned examples before or during fine-tuning .
Outcome:	The proposed method outperforms prior detectors on two machine translation datasets and one QA dataset.

Second Language (Arabic) Acquisition of LLMs via Progressive Vocabulary Expansion (2025.acl-long)

Copied to clipboard

Jianqing Zhu, Huang Huang, Zhihang Lin, Juhao Liang, Zhengyang Tang, Khalid Almubarak, Mosen Alharthi, Bang An, Juncai He, Xiangbo Wu, Fei Yu, Junying Chen, Ma Zhuoheng, Yuhao Du, He Zhang, Saied Alshahrani, Emad A. Alghamdi, Lian Zhang, Ruoyu Sun, Haizhou Li, Benyou Wang, Jinchao Xu

Unleashing the Power of LLMs in Court View Generation by Stimulating Internal Knowledge and Incorporating External Knowledge (2024.findings-naacl)

Challenge:	In the evolving landscape of large language models, the predominant focus has been on English and Chinese.
Approach:	They propose to utilize Arabic-specific vocabulary in the tokenizer to accelerate decoding.
Outcome:	The proposed model achieves decent performance comparable to the best Arabic LLMs across various Arabic benchmarks.

Copied to clipboard

Yifei Liu, Yiquan Wu, Ang Li, Yating Zhang, Changlong Sun, Weiming Lu, Fei Wu, Kun Kuang

DeepSolution: Boosting Complex Engineering Solution Design via Tree-based Exploration and Bi-point Thinking (2025.acl-long)

Challenge:	Large Language Models (LLMs) have made remarkable strides in language generation, but they encounter difficulties in the knowledge-intensive legal domain.
Approach:	They propose to decompose court views into different parts, stimulate internal knowledge, and incorporate external information to unleash the power of LLMs in the task.
Outcome:	The proposed method generates more accurate and reliable court views on two real-world datasets LAIC2021 and CJO2022.

Copied to clipboard

Zhuoqun Li, Haiyang Yu, Xuanang Chen, Hongyu Lin, Yaojie Lu, Fei Huang, Xianpei Han, Yongbin Li, Le Sun

Challenge:	Existing studies in retrieval-augmented generation (RAG) do not sufficiently address the design of complex engineering solutions.
Approach:	They propose a retrieval-augmented generation system that leverages tree-based exploration and bi-point thinking mechanism to generate reliable solutions.
Outcome:	Experiments show that the proposed system achieves state-of-the-art (SOTA) performance on the SolutionBench, highlighting its potential to enhance the automation and reliability of complex engineering solution design in real-world applications.

RexUIE: A Recursive Method with Explicit Schema Instructor for Universal Information Extraction (2023.findings-emnlp)

Copied to clipboard

Chengyuan Liu, Fubang Zhao, Yangyang Kang, Jingyuan Zhang, Xiang Zhou, Changlong Sun, Kun Kuang, Fei Wu

Information Extraction & Text Mining

Challenge:	Named Entity Recognition (NER) and Relation Extraction (RE) models have limited success when extracting general schemas such as quadruples and quintuples.
Approach:	They propose a formal formulation that covers almost all extraction schemas and a Recursive Method with Explicit Schema Instructor for UIE.
Outcome:	The proposed method shows strong performance under full-shot and few-shot settings and achieves state-of-the-art results on the tasks of extracting complex schemas.

InductionBench: LLMs Fail in the Simplest Complexity Class (2025.acl-long)

Copied to clipboard

Wenyue Hua, Tyler Wong, Fei Sun, Liangming Pan, Adam Jardine, William Yang Wang

Adversarial Knowledge Stimulated Contrastive Prompting for Few-shot Language Learners (2023.findings-acl)

Challenge:	Existing benchmarks focus on deductive reasoning, largely overlooking inductive reasoning.
Approach:	They propose a benchmark to evaluate the inductive reasoning ability of large language models.
Outcome:	The proposed benchmark demonstrates that even the most advanced modelw struggle to master the simplest complexity classes within the subregular hierarchy of functions.

Copied to clipboard

Kai Zheng, Qingfeng Sun, Yaming Yang, Tengchao Lv, Yeyong Pi, Changlin Zhao, Fei Xu, Qi Zhang

When to Trust LLMs: Aligning Confidence with Response Quality (2024.findings-acl)

Challenge:	Prompt-based fine-tuning has boosted performance of Pre-trained language models on few-shot Natural Language Understanding (NLU) tasks by employing task-specific prompts.
Approach:	They propose a Cloze-driven prompt framework for prompt tuning that implicitly stimulates knowledge from pre-trained language models.
Outcome:	The proposed framework outperforms state-of-the-art for prompt-based fine-tuning on few-shot NLU tasks.

Copied to clipboard

Shuchang Tao, Liuyi Yao, Hanxing Ding, Yuexiang Xie, Qi Cao, Fei Sun, Jinyang Gao, Huawei Shen, Bolin Ding

Challenge:	Existing methods express reliability by confidence level, but lack objective guidance . Existing approaches express reliability but lack guidance on when to trust LLMs .
Approach:	They propose a reward-based approach to align confidence with quality to ensure reliability . they propose 'conqORD' to help model to verbalize greater confidence for higher quality responses .
Outcome:	Experiments show that CONQORD significantly improves confidence and response accuracy . the proposed approach can be used to determine reliability of large language models .

More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs (2024.emnlp-main)

Copied to clipboard

Chengyuan Liu, Yangyang Kang, Shihang Wang, Lizhi Qing, Fubang Zhao, Chao Wu, Changlong Sun, Kun Kuang, Fei Wu

XFormParser: A Simple and Effective Multimodal Multilingual Semi-structured Form Parser (2025.coling-main)

Challenge:	a recent study shows that performance on general tasks decreases after Large Language Models are fine-tuned on domain-specific tasks.
Approach:	They propose a general capability integration approach to integrate general capabilities and domain knowledge within a single instance.
Outcome:	The proposed method improves performance on domain-specific tasks by integrating general capabilities and domain knowledge.

Copied to clipboard

Xianfu Cheng, Hang Zhang, Jian Yang, Xiang Li, Weixiao Zhou, Fei Liu, Kui Wu, Xiangyuan Guan, Tao Sun, Xianjie Wu, Tongliang Li, Zhoujun Li

Multilinguality

Challenge:	Document AI parsing semi-structured image form is a key information extraction task.
Approach:	They propose a multimodal and multilingual semi-structured FORM PARSER which integrates SER and relation extraction into a unified framework.
Outcome:	The proposed framework achieves up to 1.79% improvement on RE tasks in multilingual and zero-shot settings.

kFolden: k-Fold Ensemble for Out-Of-Distribution Detection (2021.emnlp-main)

Copied to clipboard

Xiaoya Li, Jiwei Li, Xiaofei Sun, Chun Fan, Tianwei Zhang, Fei Wu, Yuxian Meng, Jun Zhang

Text Classification

Challenge:	Existing studies studying OOD detection in NLP often rely on external data to diversify model predictions.
Approach:	They propose a framework which mimics OOD detection behavior without external data . they take text classification as an archetype and compare them to existing datasets .
Outcome:	The proposed framework can resolve in- and out-distribution examples in a natural way using existing datasets.

Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models (2025.acl-long)

Copied to clipboard

Fei Wang, Xingchen Wan, Ruoxi Sun, Jiefeng Chen, Sercan O Arik

Multimodal Dialogue Response Generation (2022.acl-long)

Challenge:	Existing studies have not linked the behavior of retrieval augmented generation (RAG) with imperfect retrieval, including irrelevant, misleading, or even malicious information.
Approach:	They propose an approach that integrates external knowledge with source-awareness to overcome imperfect retrieval errors in RAG.
Outcome:	The proposed approach is superior to previous robustness-enhanced approaches under the worst-case scenario.

Copied to clipboard

Qingfeng Sun, Yujing Wang, Can Xu, Kai Zheng, Yaming Yang, Huang Hu, Fei Xu, Jessica Zhang, Xiubo Geng, Daxin Jiang

Cross-Lingual Contrastive Learning for Fine-Grained Entity Typing for Low-Resource Languages (2022.acl-long)

Challenge:	Existing studies focus on multimodal dialogue models but neglect generation methods.
Approach:	They propose a multimodal dialogue response generation task which requires multimodal dialogs containing both texts and images which are difficult to obtain.
Outcome:	Experiments show that the proposed model can generate informative text and high-resolution image responses.

Copied to clipboard

Xu Han, Yuqi Luo, Weize Chen, Zhiyuan Liu, Maosong Sun, Zhou Botong, Hao Fei, Suncong Zheng

Fast Nearest Neighbor Machine Translation (2022.findings-acl)

Challenge:	Experimental results show that by applying our framework, we can easily learn effective FGET models for low-resource languages.
Approach:	They propose a cross-lingual contrastive learning framework to learn FGET models for low-resource languages.
Outcome:	The proposed framework can learn effective FGET models for low-resource languages even without human-labeled data.

Copied to clipboard

Yuxian Meng, Xiaoya Li, Xiayu Zheng, Fei Wu, Xiaofei Sun, Tianwei Zhang, Jiwei Li

Machine Translation

Challenge:	Fast kNN-MT uses the entire corpus as the datastore for the nearest neighbor search . knn-MT is two-orders slower than vanilla MT models .
Approach:	They propose a fast kNN-MT model that uses the entire corpus as the datastore for nearest neighbor search.
Outcome:	The proposed model is two-orders faster than kNN-MT and is only two times slower than the standard model.

MedConQA: Medical Conversational Question Answering System based on Knowledge Graphs (2022.emnlp-demos)

Copied to clipboard

Fei Xia, Bin Li, Yixuan Weng, Shizhu He, Kang Liu, Bin Sun, Shutao Li, Jun Zhao

Natural Language Interfaces

Challenge:	Existing medical dialogue systems have the problems of weak scalability, insufficient knowledge, and poor controllability.
Approach:	They propose a medical conversational question-answering system based on the knowledge graph to improve scalability and controllability.
Outcome:	The proposed system can conduct knowledge-grounded dialogues with users, using a Chinese medical knowledge graph and a large-scale dataset.

Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration (2023.emnlp-main)

Copied to clipboard

Yiquan Wu, Siying Zhou, Yifei Liu, Weiming Lu, Xiaozhong Liu, Yating Zhang, Changlong Sun, Fei Wu, Kun Kuang

Green & Sustainable NLP

Challenge:	Recent advances in deep learning have enabled a variety of techniques to be used to solve the LJP task.
Approach:	They propose a framework that leverages the strength of both LLMs and domain-specific models in the context of precedents.
Outcome:	The proposed framework leverages the strength of both LLM and domain models in the context of precedents.

VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism (2025.acl-long)

Copied to clipboard

Congzhi Zhang, Jiawei Peng, Zhenglin Wang, Yilong Lai, Haowen Sun, Heng Chang, Fei Ma, Weijiang Yu

Multimodality

Challenge:	Large Vision-Language Models (LVLMs) have shown exceptional performance in multimodal tasks, but their effectiveness in complex visual reasoning is constrained.
Approach:	They propose a training-free approach that enhances Reasoning in Large Vision-Language Models . they propose integrating Monte Carlo Tree Search and Self-Reward mechanisms into the reasoning tree .
Outcome:	The proposed approach surpasses current prompting methods and secures state-of-the-art performance across three multimodal reasoning benchmarks.

Jailbreak LLMs through Internal Stance Manipulation (2025.emnlp-main)

Copied to clipboard

Shuangjie Fu, Du Su, Beining Huang, Fei Sun, Jingang Wang, Wei Chen, Huawei Shen, Xueqi Cheng

Geoparsing: Diagram Parsing for Plane and Solid Geometry with a Unified Formal Language (2026.findings-acl)

Challenge:	Existing approaches to exploit LLMs' inherent safety mechanism, including GCG and AutoDAN, are ineffective for certain malicious requests.
Approach:	They propose a method that generates jailbreak prompts to suppress a refusal stance and induce affirmative responses by modifying adversarial prompts.
Outcome:	The proposed method outperforms the best baseline approach in Llama-2-7b-chat and achieves a 92.2% success rate across all models.

Copied to clipboard

Peijie Wang, Ming-Liang Zhang, Jun Cao, Chao Deng, Dekang Ran, Pi Bu, Hongda Sun, Xuan Zhang, Yingyao Wang, Jun Song, Bo Zheng, Fei Yin, Cheng-Lin Liu

DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings (2021.emnlp-main)

Challenge:	Recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities across various vision reasoning tasks.
Approach:	They propose a unified formal language that integrates plane and solid geometry, comprehensively covering geometric structures and semantic relations.
Outcome:	The proposed language achieves state-of-the-art parsing performance and significantly boosts MLLMs’ capabilities for downstream geometry reasoning tasks.

Copied to clipboard

Che Liu, Rui Wang, Jinghua Liu, Jian Sun, Fei Huang, Luo Si

Towards Fewer Hallucinations in Knowledge-Grounded Dialogue Generation via Augmentative and Contrastive Knowledge-Dialogue (2023.acl-short)

Challenge:	Conventional approaches to learning sentence embeddings from dialogues employ the siamese-network for this task, but such architecture yields a large gap between training and evaluating.
Approach:	They propose a dialogue-based contrastive learning approach to learn sentence embeddings from dialogues using a siamese-network.
Outcome:	The proposed model outperforms baseline methods on three multi-turn dialogue datasets in terms of MAP and Spearman’s correlation measures.

Copied to clipboard

Bin Sun, Yitong Li, Fei Mi, Fanhu Bie, Yiwei Li, Kan Li

Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)

Challenge:	Existing knowledge-grounded dialogue generation models face the hallucination problem . Existing models generate inappropriate knowledge and generate inconsistent responses .
Approach:	They propose an Augmentative and Contrastive Knowledge Dialogue Expansion Framework to enhance existing knowledge dialogue models by polarizing optimization objectives and weak knowledge generation ability.
Outcome:	The proposed framework expands existing training sets and smooths the optimization objective that enables models to generate ground-truth with or without gold knowledge.

Copied to clipboard

Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, Xin Chen, Jingang Wang, Xunliang Cai, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng

Challenge:	Existing detection methods fail to account for self-consistent error . study identifies self-consistency errors and evaluates them .
Approach:	They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors.
Outcome:	The proposed method significantly enhances performance on self-consistent errors across three LLM families.

Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts? (2024.acl-long)

Copied to clipboard

Hexiang Tan, Fei Sun, Wanli Yang, Yuanzhuo Wang, Qi Cao, Xueqi Cheng

Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models (2025.findings-acl)

Challenge:	Recent advances in augmenting Large Language Models (LLMs) with auxiliary information have significantly revolutionized their efficacy in knowledge-intensive tasks.
Approach:	They propose a systematic framework to identify whether LLMs’ responses are attributed to either generated or retrieved contexts.
Outcome:	The proposed framework identifies whether LLMs’ responses are attributed to either generated or retrieved contexts.

Copied to clipboard

Qingyu Ren, Jie Zeng, Qianyu He, Jiaqing Liang, Yanghua Xiao, Weikang Zhou, Zeye Sun, Fei Yu

Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Models (2024.acl-long)

Challenge:	In real-world scenarios, user instructions often contain soft constraints, which are semantically related and cannot be rule-based verified, posing challenges for large language models.
Approach:	They propose a pipeline to construct datasets with high-quality outputs for instructions containing soft constraints automatically and use Direct Preference Optimization (DPO) as the training method.
Outcome:	The proposed model improves the LLMs' soft constraint following ability by using direct preference optimization (DPO) and constraint quantity.

Copied to clipboard

Fangzhi Xu, Zhiyong Wu, Qiushi Sun, Siyu Ren, Fei Yuan, Shuai Yuan, Qika Lin, Yu Qiao, Jun Liu

The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse (2024.findings-acl)

Challenge:	Large Language Models (LLMs) have limitations when it comes to comprehending and expressing world knowledge that extends beyond the boundaries of natural language.
Approach:	They propose a model that integrates symbolic data into LLM training without loss of generality ability.
Outcome:	The proposed model performs better on symbol- and NL-centric tasks.

Copied to clipboard

Wanli Yang, Fei Sun, Xinyu Ma, Xun Liu, Dawei Yin, Xueqi Cheng

ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information (2021.acl-long)

Challenge:	Even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks.
Approach:	They propose to use perplexity as a surrogate metric to determine whether an edited model's performance is affected by a single edit.
Outcome:	The proposed method shows that even a single edit can cause model collapse, manifesting as significant performance degradation in various benchmark tasks.

Copied to clipboard

Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng, Xiang Ao, Qing He, Fei Wu, Jiwei Li

MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions (2023.acl-long)

Challenge:	ChineseBERT model incorporates glyph and pinyin information of Chinese characters into pretraining . proposed model achieves new performance boost over baseline models with fewer training steps .
Approach:	They propose a ChineseBERT model that incorporates glyph and pinyin information into pretraining . the glyph embedding is obtained based on different fonts of a character, and the pinyink embeddment characterizes the pronunciation of Chinese characters.
Outcome:	The proposed model achieves new performance boosts over baseline models with fewer training steps.

Copied to clipboard

Hao Sun, Zhexin Zhang, Fei Mi, Yasheng Wang, Wei Liu, Jianwei Cui, Bin Wang, Qun Liu, Minlie Huang

Natural Language Interfaces

Challenge:	A moral dialogue system aligned with users’ values could enhance conversation engagement and user connections.
Approach:	They propose a framework to train and evaluate moral dialogue systems based on communication mechanisms of morality and a method to construct moral discussions between simulated users and the dialogue system.
Outcome:	The proposed framework can train and evaluate moral dialogue systems based on simulated users and their values .

Instructions are all you need: Self-supervised Reinforcement Learning for Instruction Following (2026.acl-long)

Copied to clipboard

Qingyu Ren, Qianyu He, Powei Chang, Jie Zeng, Zeye Sun, Fei Yu, Jiaqing Liang, Yanghua Xiao

Multi-Scale Distribution Deep Variational Autoencoder for Explanation Generation (2022.findings-acl)

Challenge:	Existing reinforcement learning approaches suffer from dependency on external supervision and sparse reward signals from multi-constraint tasks.
Approach:	They propose a self-supervised reinforcement learning framework that eliminates dependency on external supervision by deriving reward signals directly from instructions and generating pseudo-labels for reward model training.
Outcome:	The proposed framework achieves strong improvements across 3 in-domain and 5 out-of-domain datasets while maintaining computational efficiency.

Copied to clipboard

ZeFeng Cai, Linlin Wang, Gerard de Melo, Fei Sun, Liang He

Explainability & Interpretability in NLP

Challenge:	Existing methods for generating explanations for recommender systems produce generic explanations that fail to incorporate user and item specific details.
Approach:	They propose a multi-scale distribution deepvariational autoencoder with a prior network that eliminates noise while retaining meaningful signals in the input.
Outcome:	The proposed models can generate explanations with concrete input-specific contents.

Paraphrase Generation as Unsupervised Machine Translation (2022.coling-1)

Copied to clipboard

Xiaofei Sun, Yufei Tian, Yuxian Meng, Nanyun Peng, Fei Wu, Jiwei Li, Chun Fan

Retrieved In-Context Principles from Previous Mistakes (2024.emnlp-main)

Challenge:	Existing methods for paraphrase generation rely on labeled datasets or are limited in narrow domains.
Approach:	They propose a paradigm for paraphrase generation by treating the task as unsupervised machine translation based on pairs of unlabeled monolingual sentences.
Outcome:	The proposed paradigm can generate paraphrases on a large unlabeled monolingual corpus without relying on bilingual sentence pairs.

Copied to clipboard

Hao Sun, Yong Jiang, Bo Wang, Yingyan Hou, Yan Zhang, Pengjun Xie, Fei Huang

BertGCN: Transductive Text Classification by Combining GNN and BERT (2021.findings-acl)

Challenge:	Recent advances in in-context learning (ICL) have limited customization and inadequate error coverage.
Approach:	They propose a method to retrieve in-context principles from mistakes to improve model performance.
Outcome:	The proposed framework enhances model performance when applied to various prompting strategies.

Copied to clipboard

Yuxiao Lin, Yuxian Meng, Xiaofei Sun, Qinghong Han, Kun Kuang, Jiwei Li, Fei Wu

Information Retrieval

Challenge:	Text classification is a core task in natural language processing (NLP) Graph neural networks (GNNs) serve as an effective approach for transductive learning.
Approach:	They propose a model that combines large scale pretraining and transductive learning for text classification.
Outcome:	The proposed model achieves SOTA performance on a wide range of datasets.

Rethinking Denoised Auto-Encoding in Language Pre-Training (2021.emnlp-main)

Copied to clipboard

Fuli Luo, Pengcheng Yang, Shicheng Li, Xuancheng Ren, Xu Sun, Songfang Huang, Fei Huang

Low-Entropy Watermark Detection via Bayes’ Rule Derived Detector (2025.findings-acl)

Challenge:	Pre-trained models such as BERT have achieved success in learning sequence representations, but they tend to learn representations that are covariant with the noise of pre-training.
Approach:	They propose to train self-trained models to learn noise invariant sequence representations . they encourage consistency between original sequence and corrupted version via unsupervised instance-wise training signals.
Outcome:	The proposed model improves on 11 natural language understanding and cross-modal tasks and achieves 0.6% gain on GLUE benchmarks and 0.8% increment on NLVR2 .

Copied to clipboard

Beining Huang, Du Su, Fei Sun, Qi Cao, Huawei Shen, Xueqi Cheng

Numerical Reasoning

Challenge:	Existing methods for text watermarking ignore strong evidences embedded in low-entropy tokens, causing statistical measures to falsely indicate the absence of a watermark.
Approach:	They propose a Bayes' Rule derived watermark Detector which exploits watermark information from every token by leveraging the posterior probability of watermark’s presence.
Outcome:	The proposed method achieves 50% and 70% relative improvements over baselines in code generation and math problem-solving tasks.

MPBoCo: Multimodal Prompt-based Boundary-enhanced Continual Framework for Joint Entity and Relation Extraction (2026.acl-long)

Copied to clipboard

Guanglu Sun, Xinyu Liu, Lili Liang, Yang Yu, Fei Lang, Suxia Zhu, Ming Liu

The Evolution of Thought: Tracking LLM Overthinking via Reasoning Dynamics Analysis (2026.acl-long)

Challenge:	Existing methods struggle to balance real-time adaptability and computational efficiency in continual learning scenarios.
Approach:	They propose a Continual Multimodal Entity and Relation Joint Extraction task and a Multimodal Prompt-based Boundary-enhanced Continuum framework that stores task-specific knowledge via learnable multimodal prompts.
Outcome:	The proposed framework outperforms baseline methods in real-world scenarios by 5.5% and 7.2%.

Copied to clipboard

Zihao Wei, Liang Pang, Jiahao Liu, Wenjie Shi, Jingcheng Deng, Shicheng Xu, Zenghao Duan, Jingang Wang, Fei Sun, Huawei Shen, Xueqi Cheng

Estimating Soft Labels for Out-of-Domain Intent Detection (2022.emnlp-main)

Challenge:	Explicit reasoning trajectories increase performance but often trigger overthinking . despite its importance, this study examines how each step of reasoning affects the final outcome .
Approach:	They propose a Reasoning Completion Point Detector that detects the RCP by monitoring rank dynamics of termination tokens.
Outcome:	The proposed method reduces token usage by up to 44% while preserving accuracy.

Copied to clipboard

Hao Lang, Yinhe Zheng, Jian Sun, Fei Huang, Luo Si, Yongbin Li

Sentiment Analysis

Challenge:	Existing methods to detect out-of-dominance (OOD) intents are limited by the lack of OOD samples.
Approach:	They propose an adaptive soft pseudo labeling method that can estimate soft labels for pseudo OOD samples when training OOD detectors.
Outcome:	The proposed method outperforms competing methods on three benchmark datasets and consistently outperformed previous methods.

GPT-NER: Named Entity Recognition via Large Language Models (2025.findings-naacl)

Copied to clipboard

Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, Guoyin Wang, Chen Guo

AceGPT, Localizing Large Language Models in Arabic (2024.naacl-long)

Challenge:	Large-scale language models (LLMs) have shown impressive ability for in-context learning with limited training data.
Approach:	They propose a novel sequence labeling task that transforms a sequence labeled as a text-generation task into a self-verification task that LLMs can adapt to.
Outcome:	The proposed model performs better on NER than supervised models on a variety of tasks . the proposed model can be easily adapted by LLMs to generate a text sequence .

Copied to clipboard

Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Song Dingjie, Zhihong Chen, Mosen Alharthi, Bang An, Juncai He, Ziche Liu, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, Jinchao Xu

Legal Judgment Prediction based on Knowledge-enhanced Multi-Task and Multi-Label Text Classification (2025.naacl-long)

Challenge:	Significant concerns emerge when addressing cultural sensitivity and local values.
Approach:	They propose a localized Large Language Model (LLM) specifically for Arabic, a language imbued with unique cultural characteristics inadequately addressed by current mainstream models.
Outcome:	The proposed model sets the state-of-the-art standard for open Arabic LLMs across various benchmarks.

Copied to clipboard

Ang Li, Yiquan Wu, Ming Cai, Adam Jatowt, Xiang Zhou, Weiming Lu, Changlong Sun, Fei Wu, Kun Kuang

Advancing Sequential Numerical Prediction in Autoregressive Models (2025.acl-short)

Challenge:	Legal judgment prediction (LJP) is an essential task for legal AI, aiming at predicting judgments based on the facts of a case.
Approach:	They propose a knowledge-enhanced approach that incorporates 'label-level knowledge' to enhance the representation of case facts for each task and 'task-level' knowledge to improve synergy.
Outcome:	The proposed method is effective in comparison to state-of-the-art (SOTA) baselines.

Copied to clipboard

Xiang Fei, Jinghui Lu, Qi Sun, Hao Feng, Yanjie Wang, Wei Shi, An-Lan Wang, Jingqun Tang, Can Huang

Challenge:	Autoregressive models are the de facto choice for sequence generation tasks, but standard approaches treat digits as independent tokens and apply cross-entropy loss, overlooking the coherent structure of numerical sequences.
Approach:	They propose a novel approach to entropy loss by extending the Earth Mover’s Distance to preserve ordinal relationships between numerical values and sequence-level to penalize the overall discrepancy between predicted and actual sequences.
Outcome:	Extensive experiments show that NTIL improves numerical prediction and integrates effectively with LLMs/MLLMs.

A Synthetic Data Generation Framework for Grounded Dialogues (2023.acl-long)

Copied to clipboard

Jianzhu Bao, Rui Wang, Yasheng Wang, Aixin Sun, Yitong Li, Fei Mi, Ruifeng Xu

MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes (2025.findings-acl)

Challenge:	Existing approaches to train grounded dialogues require large amounts of data.
Approach:	They propose a synthetic data generation framework for grounded dialogues that takes knowledge data and heuristics to determine a dialogue flow and incrementally turn it into a dialog.
Outcome:	The proposed framework significantly boosts model performance in training data and low-resource scenarios.

Copied to clipboard

Asma Ben Abacha, Wen-wai Yim, Yujuan Fu, Zhaoyi Sun, Meliha Yetisgen, Fei Xia, Thomas Lin

Layer-wise Model Pruning based on Mutual Information (2021.emnlp-main)

Challenge:	Several studies have shown that large language models can answer medical questions correctly, outperforming the average human score in some medical exams.
Approach:	They introduce MEDEC, the first publicly available benchmark for medical error detection and correction in clinical notes.
Outcome:	The proposed model outperforms medical doctors in errors detection and correction tasks.

Copied to clipboard

Chun Fan, Jiwei Li, Tianwei Zhang, Xiang Ao, Fei Wu, Yuxian Meng, Xiaofei Sun

Focus-aware Response Generation in Inquiry Conversation (2023.findings-acl)

Challenge:	In spite of impressive results of neural networks, the huge model size has hindered their applications in cases where computation and memory resources are limited.
Approach:	They propose a method for layer-wise pruning using mutual information based feature selection in SVMs and logistic regression.
Outcome:	The proposed pruning strategy offers greater speedup and higher performance than weight-based pruning methods.

Copied to clipboard

Yiquan Wu, Weiming Lu, Yating Zhang, Adam Jatowt, Jun Feng, Changlong Sun, Fei Wu, Kun Kuang

Pruning Foundation Models for High Accuracy without Retraining (2024.findings-emnlp)

Challenge:	Existing studies on response generation focus on relevance and fluency, rarely paying attention to the focus.
Approach:	They propose a Focus-aware response generation method that takes the focus into consideration and optimizes a multi-level encoder and focal decoder to generate multiple candidate responses.
Outcome:	The proposed method generates candidate responses that correspond to different focuses and performs better on two orthogonal inquiry conversation datasets.

Copied to clipboard

Pu Zhao, Fei Sun, Xuan Shen, Pinrui Yu, Zhenglun Kong, Yanzhi Wang, Xue Lin

Xiaofei Sun, Yuxian Meng, Xiang Ao, Fei Wu, Tianwei Zhang, Jiwei Li, Chun Fan

Challenge:	Despite the superior performance of foundation models, it is challenging to deploy large language models in practical applications due to their massive parameters and computations.
Approach:	They propose a pruning algorithm to prune LLMs in one-shot without retraining . they propose retrainable pruning algorithms to prune multiple weights in LLM .
Outcome:	The proposed pruning methods perform better than baseline pruning methods on sparse and unstructured sparsity models.

Copied to clipboard

EviReport: From Reasoned Outlines to Evidence Tracked Long-Form Reports (2026.findings-acl)

Challenge:	Existing methods to measure sentence similarity face limited dataset size and training-test gap . existing methods lack large-scale labeled datasets with labeles that are labor-intensive and expensive .
Approach:	They propose a framework that measures sentence similarity by comparing probabilities of generating two sentences given the same context.
Outcome:	The proposed framework achieves significant performance boosts over baselines under supervised and unsupervised settings.

Copied to clipboard

Zihan Liu, Jianhui li, Zexin Wang, Fei Sun, Jingjing LI, Zheyuan Li, Ke Xiang, Hang Cui, Houhua Gong, Changhua Pei, Gaogang Xie

Multimodality

Challenge:	Evidence-intensive reports often produce fluent but under-supported drafts . eviReport is an evidence-grounded workflow for automated long-form report generation .
Approach:	They propose an evidence-tracked workflow that organizes corpus evidence into compact, traceable units and retrieves query-relevant subgraphs into retrieval-ready packages.
Outcome:	The proposed workflow outperforms baselines in factual coverage, factual accuracy and visual evidence integration.

Order Matters: Investigate the Position Bias in Multi-constraint Instruction Following (2025.findings-acl)

Copied to clipboard

Jie Zeng, Qianyu He, Qingyu Ren, Jiaqing Liang, Weikang Zhou, Zeye Sun, Fei Yu, Yanghua Xiao

Challenge:	Existing large language models struggle to follow multi-constraint instructions in real-world applications.
Approach:	They propose to quantify the difficulty distribution of constraints by a novel Difficulty Distribution Index (CDDI) they find that LLMs are more performant when presented with constraints in a “hard-to-easy” order.
Outcome:	The proposed model is more performant when presented with constraints in a “hard-to-easy” order, compared with existing models with different architectures and sizes of parameters.

Text Classification via Large Language Models (2023.findings-emnlp)

Copied to clipboard

Xiaofei Sun, Xiaoya Li, Jiwei Li, Fei Wu, Shangwei Guo, Tianwei Zhang, Guoyin Wang

SimpleDeepSearcher: Deep Information Seeking via Web-Powered Reasoning Trajectory Synthesis (2025.findings-emnlp)

Challenge:	Large-scale Language Models (LLMs) have shown the ability for in-context learning.
Approach:	They propose a progressive reasoning strategy tailored to addressing complex linguistic phenomena such as intensification, contrast, irony and limited number of tokens allowed in in-context learning.
Outcome:	The proposed model performs better on 4 out of 5 widely-used text-classification benchmarks, while demonstrating comparable performance to SOTA on MR.

Copied to clipboard

Shuang Sun, Huatong Song, Yuhao Wang, Ruiyang Ren, Jinhao Jiang, Junjie Zhang, Fei Bai, Jia Deng, Xin Zhao, Zheng Liu, Lei Fang, Zhongyuan Wang, Ji-Rong Wen

Dice Loss for Data-imbalanced NLP Tasks (2020.acl-main)

Challenge:	Existing approaches to deep search training lack high-quality training trajectories, prohibitive computational costs and lack of high-fidelity training data.
Approach:	They propose a framework that synthesizes high-quality training data by simulating real user interactions in live web search environments.
Outcome:	The proposed framework synthesizes high-quality training data by simulating user interactions in live web search environments.

Copied to clipboard

Xiaoya Li, Xiaofei Sun, Yuxian Meng, Junjun Liang, Fei Wu, Jiwei Li

Syntactic Text Processing

Challenge:	Using dice loss, we find that data imbalance is a common issue in many NLP tasks . data imbalance affects the performance of many tasks, such as tagging and machine reading comprehension .
Approach:	They propose to use dice loss to replace the standard cross-entropy objective for data-imbalanced NLP tasks.
Outcome:	The proposed training objective achieves significant performance boost on a wide range of data imbalanced tasks.

COLD: A Benchmark for Chinese Offensive Language Detection (2022.emnlp-main)

Copied to clipboard

Jiawen Deng, Jingyan Zhou, Hao Sun, Chujie Zheng, Fei Mi, Helen Meng, Minlie Huang

Ethical NLP

Challenge:	Offensive language detection is crucial for maintaining a civilized social media platform and deploying pre-trained language models.
Approach:	They propose a benchmark benchmark for Chinese offensive language analysis including a Chinese Offensive Language Dataset and a baseline detector which is trained on the dataset.
Outcome:	The proposed benchmark contributes to Chinese offensive language detection which is challenging for existing resources.

The Fall of ROME: Understanding the Collapse of LLMs in Model Editing (2024.findings-emnlp)

Copied to clipboard

Wanli Yang, Fei Sun, Jiajun Tan, Xinyu Ma, Du Su, Dawei Yin, Huawei Shen

BaseCal: Unsupervised Confidence Calibration via Base Model Signals (2026.acl-long)

Challenge:	Recent studies have found that model editing methods can cause large language models to collapse with just a single edit.
Approach:	They propose a method that uses prefixed keys and adds prefixes during testing to prevent model collapse.
Outcome:	The proposed method prevents model collapse while maintaining effectiveness, the authors show . Rank-One Model Editing (ROME) has been found to cause model collapse with just a single edit .

Copied to clipboard

Hexiang Tan, Wanli Yang, Junwei Zhang, Xin Chen, Rui Tang, Du Su, Jingang Wang, Yuanzhuo Wang, Fei Sun, Xueqi Cheng

CodeEvo: Interaction-Driven Synthesis of Code-centric Data through Hybrid and Iterative Feedback (2026.acl-long)

Challenge:	Post-trained LLMs typically compromise reliability with severe overconfidence, resulting in inaccurate responses.
Approach:	They propose a solution that feeds PoLLMs into the base LLM to get confidence.
Outcome:	The proposed solution reduces expected calibration error (ECE) by 42.90% compared to the best unsupervised baselines.

Copied to clipboard

Qiushi Sun, Jingyang Gong, Lei Li, Qipeng Guo, Fei Yuan

Challenge:	Existing methods for generating instruction-code pairs rely on rigid heuristics and are labor-intensive.
Approach:	They propose a dual-agent architecture that integrates a Coder and a Reviewer to orchestrate the generation trajectory.
Outcome:	The proposed architecture outperforms baselines on a large-scale dataset of instruction-code pairs with stepped difficulty levels.

Large Language Models are Better Reasoners with Self-Verification (2023.findings-emnlp)

Copied to clipboard

Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, Jun Zhao

Dynamic Stochastic Decoding Strategy for Open-Domain Dialogue Generation (2024.findings-acl)

Challenge:	Existing methods to solve complex natural language processing tasks require multiple steps to verify the answers.
Approach:	They propose to use chain of thought prompting to solve reasoning tasks with large language models.
Outcome:	The proposed method can improve reasoning performance on arithmetic, commonsense, and logical reasoning datasets.

Copied to clipboard

Yiwei Li, Fei Mi, Yitong Li, Yasheng Wang, Bin Sun, Shaoxiong Feng, Kan Li

Combating Security and Privacy Issues in the Era of Large Language Models (2024.naacl-tutorials)

Challenge:	Stochastic sampling strategies are not widely used in open-domain dialogue systems.
Approach:	They propose a dynamic decoding strategy which can adjust the decoding space w.r.t. different contexts.
Outcome:	The proposed decoding strategy can improve the performance of pre-trained models when coupled with four well-used stochastic decoding algorithms.

Copied to clipboard

Muhao Chen, Chaowei Xiao, Huan Sun, Lei Li, Leon Derczynski, Anima Anandkumar, Fei Wang