Papers by Yu Gu

LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)

Challenge:	Test-time computing approaches that leverage additional computational resources during inference have been proven effective in enhancing large language model performance.
Approach:	They propose a linearly scaling approach that leverages local consistency of neighboring unlabeled data to improve test-time predictions.
Outcome:	The proposed approach outperforms baseline methods such as prompting and self-consistency across eight datasets and performs robustly across embedding models.

Copied to clipboard

Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin

Challenge:	a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities .
Approach:	They present a comparative analysis to identify and distinguish LLM activities from human activities.
Outcome:	The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities.

EvoEdit: Evolving Null-space Alignment for Robust and Efficient Knowledge Editing (2026.findings-acl)

Copied to clipboard

Sicheng Lyu, Yu Gu, Xinyu Wang, Jerry Huang, Sitao Luan, Yufei Cui, Xiao-Wen Chang, Peng Lu

An Information Bottleneck Perspective for Effective Noise Filtering on Retrieval-Augmented Generation (2024.acl-long)

Challenge:	Existing approaches to modifying large language models require continual updates to rectify outdated or erroneous knowledge.
Approach:	They propose a model editing strategy that mitigates catastrophic interference through sequential null-space alignment.
Outcome:	EvoEdit achieves better or comparable performance than prior state-of-the-art techniques with up to 3.53 speedup.

Copied to clipboard

Kun Zhu, Xiaocheng Feng, Xiyuan Du, Yuxuan Gu, Weijiang Yu, Haotian Wang, Qianglong Chen, Zheng Chu, Jingchang Chen, Bing Qin

Information Retrieval

Challenge:	Experimental results show that retrieval-augmented generation improves accuracy and relevance of large language models.
Approach:	They propose to introduce the information bottleneck theory into retrieval-augmented generation by maximizing mutual information between compression and ground output while minimizing mutual information .
Outcome:	The proposed approach improves accuracy and correctness of answer generation and conciseness with 2.5% compression rate.

Are Large Language Models Reliable Reviewers? A Benchmark for Error Detection in Financial Documents (2026.findings-acl)

Copied to clipboard

Ying He, Zhouhong Gu, Zhecheng Hu, Yubo Zhou, Hao Shen, Jiaqing Liang, Zhaoqian Dai, Ma Shuguang, Fei Yu, Yanghua Xiao, Zhixu Li

Uncertainty-Aware Routing for Principled Alignment with MoE Dynamics (2026.acl-long)

Challenge:	Existing LLMs struggle to identify errors in financial documents, a study shows . 18% of financial practitioners make errors daily, one-third make errors several times weekly, and 59% make errors multiple times monthly.
Approach:	They introduce FinED-Bench, a publicly available Benchmark for financial error detection . it covers nine real-world financial scenarios and includes over 900 documents in 2025 . supervised fine-tuning can significantly improve the performance of weaker LLMs, they show .
Outcome:	The proposed benchmark covers nine real-world financial scenarios and includes over 900 documents reported in 2025 that are unseen by existing language models.

Copied to clipboard

Yilong Chen, Junyuan Shang, Yuchen Feng, Zhenyu Zhang, Naibin Gu, Ziqi Wang, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, Haifeng Wang

Challenge:	Mixture-of-Experts (MoE) is a cornerstone for scaling LLMs, yet its training dynamics remain poorly understood, often leading to sub-optimal specialization.
Approach:	They propose to use Helmholtz Free Energy and Router Entropy to study the MoE lifecycle and identify a universal Three-Stage Phase Transition .
Outcome:	The proposed model reduces perplexity and improves expert distinctiveness, offering a principled path toward thermodynamically aligned computation.

Self-Debiasing Large Language Models: Zero-Shot Recognition and Reduction of Stereotypes (2025.naacl-short)

Copied to clipboard

Isabel O. Gallegos, Ryan Aponte, Ryan A. Rossi, Joe Barrow, Mehrab Tanjim, Tong Yu, Hanieh Deilamsalehy, Ruiyi Zhang, Sungchul Kim, Franck Dernoncourt, Nedim Lipka, Deonna Owens, Jiuxiang Gu

A Survey on LLM-based Conversational User Simulation (2026.eacl-long)

Challenge:	Large language models exhibit harmful social biases, but they are often difficult to train and modify.
Approach:	They leverage the zero-shot capabilities of large language models to reduce stereotyping . they introduce a technique called zero- shot self-debiasing to reduce bias .
Outcome:	The proposed technique reduces stereotyping across nine different social groups while relying on the LLM itself and a simple prompt.

Copied to clipboard

Bo Ni, Yu Wang, Leyao Wang, Branislav Kveton, Franck Dernoncourt, Yu Xia, Hongjie Chen, Reuben Luera, Samyadeep Basu, Subhojyoti Mukherjee, Puneet Mathur, Nesreen K. Ahmed, Junda Wu, Li Li, Huixin Zhang, Ruiyi Zhang, Tong Yu, Sungchul Kim, Jiuxiang Gu, Zhengzhong Tu, Alexa Siu, Zichao Wang, Seunghyun Yoon, Nedim Lipka, Namyong Park, Zihao Lin, Trung Bui, Yue Zhao, Tyler Derr, Ryan A. Rossi

Dialogue Systems & Conversational Agents

Challenge:	Recent advances in large language models (LLMs) have enabled high-fidelity generation of synthetic user conversation.
Approach:	They propose a taxonomy covering user granularity and simulation objectives . they analyze core techniques and evaluation methodologies to help them understand the latest developments .
Outcome:	The proposed model enables high-fidelity generation of synthetic user conversation.

JECC: Commonsense Reasoning Tasks Derived from Interactive Fictions (2023.findings-acl)

Copied to clipboard

Mo Yu, Yi Gu, Xiaoxiao Guo, Yufei Feng, Xiaodan Zhu, Michael Greenspan, Murray Campbell, Chuang Gan

PRAL: A Tailored Pre-Training Model for Task-Oriented Dialog Generation (2021.acl-short)

Challenge:	Existing benchmarks focus on a single reasoning type and ask human annotators to write candidate statements related to the particular type of commonsense.
Approach:	They propose a new commonsense reasoning dataset based on human’s Interactive Fiction (IF) gameplaywalkthroughs.
Outcome:	The proposed dataset is challenging to previous machine reading models and large language models with a significant 20%performance gap compared to human experts.

Copied to clipboard

Jing Gu, Qingyang Wu, Chongruo Wu, Weiyan Shi, Zhou Yu

Dialogue Systems & Conversational Agents

Challenge:	Existing approaches to building task-oriented dialog systems require a substantial amount of annotations and thus are labor-intensive.
Approach:	They propose a Pre-trainedRole Alternating Language model (PRAL) that is explicitly designed for task-oriented dialog tasks.
Outcome:	The proposed model outperforms or is on par with state-of-the-art models on task-oriented dialog tasks.

Revealing the Attention Floating Mechanism in Masked Diffusion Models (2026.findings-acl)

Copied to clipboard

Xin Dai, Pengcheng Huang, Zhenghao Liu, Shuo Wang, Yukun Yan, Chaojun Xiao, Yu Gu, Ge Yu, Maosong Sun

CharPoet: A Chinese Classical Poetry Generation System Based on Token-free LLM (2024.acl-demos)

Challenge:	Masked diffusion models (MDMs) leverage bidirectional attention and a denoising process.
Approach:	They investigate the attention behaviors of Masked diffusion models by revealing the phenomenon of Attention Floating.
Outcome:	The proposed model doubles the performance of autoregressive models in knowledge-intensive tasks.

Copied to clipboard

Chengyue Yu, Lei Zang, Jiaotuan Wang, Chenyi Zhuang, Jinjie Gu

FocalPO: Enhancing Preference Optimizing by Focusing on Correct Preference Rankings (2025.acl-short)

Challenge:	Traditional systems in this field usually accept keywords as user inputs, resulting in limited control over content.
Approach:	They propose a Chinese classical poetry generation system based on token-free LLMs that allow unrestricted user instructions to be used.
Outcome:	The proposed system outperforms traditional systems including Jiuge and GPT-4 in format accuracy and content quality.

Copied to clipboard

Tong Liu, Xiao Yu, Wenxuan Zhou, Jindong Gu, Volker Tresp

Challenge:	Efficient preference optimization algorithms such as Direct Preference Optimization (DPO) have become a popular approach in aligning large language models with human preferences.
Approach:	They propose a preference optimization variant that instead down-weighs misranked preference pairs and prioritizes enhancing the model’s understanding of pairs that it can already rank correctly.
Outcome:	The proposed model outperforms DPO on benchmarks like Alpaca Eval 2.0 and Arena-Hard using mistral-base-7B and Llama-3-Instruct-8B with the introduced hyperparameter fixed.

From Selection to Generation: A Survey of LLM-based Active Learning (2025.acl-long)

Copied to clipboard

Yu Xia, Subhojyoti Mukherjee, Zhouhang Xie, Junda Wu, Xintong Li, Ryan Aponte, Hanjia Lyu, Joe Barrow, Hongjie Chen, Franck Dernoncourt, Branislav Kveton, Tong Yu, Ruiyi Zhang, Jiuxiang Gu, Nesreen K. Ahmed, Yu Wang, Xiang Chen, Hanieh Deilamsalehy, Sungchul Kim, Zhengmian Hu, Yue Zhao, Nedim Lipka, Seunghyun Yoon, Ting-Hao Kenneth Huang, Zichao Wang, Puneet Mathur, Soumyabrata Pal, Koyel Mukherjee, Zhehao Zhang, Namyong Park, Thien Huu Nguyen, Jiebo Luo, Ryan A. Rossi, Julian McAuley

Label-Aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition (N18-1)

Challenge:	Large Language Models (LLMs) have been used for selection and training of data for active learning.
Approach:	They propose an intuitive taxonomy that categorizes LLM-based active learning techniques and discuss the transformative roles they can play in the active learning loop.
Outcome:	The proposed model can generate entirely new data instances and provide more cost-effective annotations with fewer labeled data instances.

Copied to clipboard

Zhenghui Wang, Yanru Qu, Liheng Chen, Jian Shen, Weinan Zhang, Shaodian Zhang, Yimei Gao, Gen Gu, Ken Chen, Yong Yu

Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)

Challenge:	NER is a fundamental problem for medical text mining because of the difference of specialties and cost of human annotation.
Approach:	They propose a label-aware double transfer learning framework for medical NER from electronic medical records.
Outcome:	The proposed framework improves accuracy over strong baselines on 12 cross-specialty NER tasks.

Copied to clipboard

Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao

Enhancing Long-Chain Reasoning Distillation through Error-Aware Self-Reflection (2026.findings-acl)

Challenge:	Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding.
Approach:	They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss .
Outcome:	The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss.

Copied to clipboard

Zhuoyang Wu, Xinze Li, Zhenghao Liu, Yukun Yan, Zhiyuan Liu, Minghe Yu, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun

Many-Shot Scaling of In-Context Learning with Self-Generated Demonstrations (2026.findings-acl)

Challenge:	Existing studies treat SLMs as student models and use long-form Chains-of-Thought (CoTs) as supervision signals for Supervised Fine-Tuning (SFT). Existing research focuses on distilling reasoning ability from LLMs to enhance the mathematical reasoning performance of small-scale models.
Approach:	They propose a framework that refines teacher CoTs through an error-aware reflection process to enable the student model to construct more tailored teacher Cots.
Outcome:	Experiments on multiple mathematical reasoning benchmarks show that ORION improves performance by more than 2% over all baselines.

Copied to clipboard

Zhengyao Gu, Henry Peng Zou, Yankai Chen, Aiwei Liu, Weizhi Zhang, Philip S. Yu

Low-Resource NLP

Challenge:	In-context learning methods that use self-generated annotations do not scale to many-shot scenarios.
Approach:	They propose a framework analogous to semi-supervised learning that uses self-generated annotations instead of ground truth labels.
Outcome:	The proposed framework outperforms ground truth ICL under zero-shot, few-shot and many-shot settings.

BLADE: Benchmarking Language Model Agents for Data-Driven Science (2024.findings-emnlp)

Copied to clipboard

Ken Gu, Ruoxi Shang, Ruien Jiang, Keying Kuang, Richard-John Lin, Donghe Lyu, Yue Mao, Youran Pan, Teng Wu, Jiaqian Yu, Yikun Zhang, Tianmai Zhang, Lanyi Zhu, Mike Merrill, Jeffrey Heer, Tim Althoff

Reimagining Safety Alignment with An Image (2025.emnlp-main)

Challenge:	Language model-based agents can be used to conduct and support data-driven science, but evaluating them on open-ended tasks is challenging due to multiple valid approaches, partially correct steps, and different ways to express the same decisions.
Approach:	They propose a benchmark to automatically evaluate agents’ multifaceted approaches to open-ended research questions.
Outcome:	BLADE evaluates agents’ multifaceted approaches to open-ended research questions using data from 12 datasets and research questions drawn from existing scientific literature.

Copied to clipboard

Yifan Xia, Guorui Chen, Wenqian Yu, Zhijiang Li, Philip Torr, Jindong Gu

Structure-Aware Language Model Pretraining Improves Dense Retrieval on Structured Data (2023.findings-acl)

Challenge:	Existing approaches to large language models face inefficiency, fragility, or architectural constraints, resulting in inefficient performance and heightened over-refusal in cross-modal tasks.
Approach:	They propose an optimization-driven visual prompt framework that enhances security and reduces over-refusal at the same time.
Outcome:	The proposed framework enhances security and reduces over-refusal while maintaining robust safety while reducing unnecessary denials.

Copied to clipboard

Xinze Li, Zhenghao Liu, Chenyan Xiong, Shi Yu, Yu Gu, Zhiyuan Liu, Ge Yu

C2PO: Diagnosing and Disentangling Bias Shortcuts in LLMs (2026.findings-acl)

Challenge:	Structure Aware Dense Retrieval (SANTA) model encodes user queries and structured data in one universal embedding space for retrieving structured data.
Approach:	They propose to use structured data and unstructured data to encode queries and structured data in one universal embedding space for retrieving structured data.
Outcome:	The proposed model achieves state-of-the-art on code search and product search and conducts convincing results in the zero-shot setting.

Copied to clipboard

Xuan Feng, Bo An, Tianlong Gu, Liang Chang, Fengrui Hao, Peipeng Yu, Shuai Zhao

MetaMem: Evolving Meta-Memory for Knowledge Utilization through Self-Reflective Symbolic Optimization (2026.findings-acl)

Challenge:	Existing approaches to solve large language models address stereotypical and structural biases in isolation . however, prior paradigms address these in isolation, often at the expense of exacerbating the other .
Approach:	They propose a framework to tackle latent spurious feature correlations within input that drive erroneous reasoning shortcuts.
Outcome:	The proposed framework mitigates stereotypical and structural biases while preserving robust general reasoning capabilities.

Copied to clipboard

Haidong Xin, Xinze Li, Zhenghao Liu, Yukun Yan, Shuo Wang, Cheng Yang, Yu Gu, Ge Yu, Maosong Sun

Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling (2026.findings-acl)

Challenge:	Existing memory systems can support long-horizon human-LLM interactions by persisting historical interactions beyond limited context windows.
Approach:	They propose a framework that augments memory systems with a self-evolving meta-memory . meta-meso is iteratively distilling transferable knowledge utilization experiences . results show MetaMem outperforms strong baselines by over 3.6% .
Outcome:	The proposed framework outperforms baselines by over 3.6% in the long-horizon human-LLM interaction.

Copied to clipboard

Shuliang Liu, Zhipeng Xu, Zhenghao Liu, Yukun Yan, Minghe Yu, Yu Gu, Chong Chen, Huiyuan Xie, Ge Yu

CompleQA: Benchmarking the Impacts of Knowledge Graph Completion Methods on Question Answering (2023.findings-emnlp)

Challenge:	Large Language Models (LLMs) are used as automatic evaluators to provide accurate and reliable assessments.
Approach:	They propose a framework that integrates LLM-based judgment models into a multi-agent system and simulates the interactive client-server polling mechanism.
Outcome:	The proposed framework outperforms supervised models trained on annotated judgment data while requiring no human-labeled annotations.

Copied to clipboard

Donghan Yu, Yu Gu, Chenyan Xiong, Yiming Yang

SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)

Challenge:	Existing studies have focused on Knowledge Graph Completion as an end in itself, neglecting its potential impact on subsequent applications.
Approach:	They propose a benchmark to assess the impact of representative KGC methods on Knowledge Graph Question Answering (KGQA) they use a knowledge graph with 3 million triplets across 5 distinct domains to evaluate their results.
Outcome:	The proposed benchmark compares four well-known methods with two state-of-the-art systems to assess the impact of incomplete knowledge graphs on KGQA.

Copied to clipboard

Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang

Memformer: A Memory-Augmented Transformer for Sequence Modeling (2022.findings-aacl)

Challenge:	Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases.
Approach:	They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs.
Outcome:	The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs.

Copied to clipboard

Qingyang Wu, Zhenzhong Lan, Kun Qian, Jing Gu, Alborz Geramifard, Zhou Yu

Don’t Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments (2023.acl-long)

Challenge:	Experimental results show that Memformer uses 8.1x less memory space and 3.2x faster on inference.
Approach:	They propose an efficient neural network that utilizes an external dynamic memory to encode and retrieve past information.
Outcome:	The proposed model achieves comparable performance against baselines with 8.1x less memory space and 3.2x faster on inference.

Copied to clipboard

Yu Gu, Xiang Deng, Yu Su

Self-Cleaning: Improving a Named Entity Recognizer Trained on Noisy Data with a Few Clean Instances (2024.findings-naacl)

Challenge:	Existing language models lack grounding to real-world environments . a missing piece is the connection between LMs and the environment .
Approach:	They propose a generic framework for grounded language understanding that capitalizes on discriminative ability of LMs instead of their generative ability.
Outcome:	The proposed framework capitalizes on discriminative ability of LMs instead of their generative ability.

Copied to clipboard

Zhendong Chu, Ruiyi Zhang, Tong Yu, Rajiv Jain, Vlad Morariu, Jiuxiang Gu, Ani Nenkova

Information Extraction & Text Mining

Challenge:	Existing methods to train named entity recognition models on noisy data are expensive and time-intensive to accumulate.
Approach:	They propose to denoise noisy NER data with guidance from a small set of clean instances.
Outcome:	The proposed method can improve on large-scale datasets with a small guidance set.

UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents (2026.acl-long)

Copied to clipboard

Yifan Ji, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Qian Zhang, Zhibo Yang, Junyang Lin, Yu Gu, Ge Yu, Maosong Sun

Information Extraction & Text Mining

Challenge:	Recent Large Multimodal Models (LMMs) have shown promising potential for performing end-to-end KIE directly from document images.
Approach:	They propose a benchmark to evaluate the performance of Large Multimodal Models (LMMs) using a constrained-category KIE track and an open-categorical KIE Track.
Outcome:	Experiments on 15 state-of-the-art LMMs show performance degradation under diverse schema definitions, long-tail key fields, and complex layouts, along with pronounced performance disparities across different document types and scenarios.

ArcaneQA: Dynamic Program Induction and Contextualized Encoding for Knowledge Base Question Answering (2022.coling-1)

Copied to clipboard

Yu Gu, Yu Su

Natural Language Interfaces

Challenge:	Existing ranking-based KBQA models struggle with flexibility in predicting complicated queries and have impractical running time.
Approach:	They propose a new generation-based question answering on knowledge bases model that addresses both large search space and ambiguities in schema linking.
Outcome:	The proposed model overcomes two intertwined challenges on popular KBQA datasets and is highly competitive and efficient.

EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning for LLMs (2026.findings-acl)

Copied to clipboard

Huanyu Liu, Jia Li, Yihong Dong, Chang Yu, Taozhi Chen, Lecheng Wang, Yongding Tao, Bin Gu, Ge Li

Chunks as Arms: Multi-Armed Bandit-Guided Sampling for Long-Context LLM Preference Optimization (2026.acl-long)

Challenge:	Existing approaches to reinforcement learning with verifiable reward (RLVR) are limited by difficulty or lack of exploration.
Approach:	They propose a self-evolving curriculum learning framework based on chain-of-thought reasoning optimization that constrains exploration space by self-generating and verifying CoT trajectories.
Outcome:	The proposed framework enables LLMs to solve previously unsolved problems without external supervision and is compatible with various RL fine-tuning methods.

Copied to clipboard

Shaohua Duan, Pengcheng Huang, Xinze Li, Zhenghao Liu, Xiaoyuan Yi, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu, Maosong Sun

On the Transformer Growth for Progressive BERT Training (2021.naacl-main)

Challenge:	Recent studies have explored fine-tuning Large Language Models with synthetic data to enhance their long-context capabilities.
Approach:	They propose a framework that leverages a Multi-Armed Bandit rollout strategy to identify the most informative chunks from the given long context for sampling high-quality and diverse responses.
Outcome:	The proposed framework achieves 4% improvement on long-context reasoning benchmarks on Llama and Qwen.

Copied to clipboard

Xiaotao Gu, Liyuan Liu, Hongkun Yu, Jing Li, Chen Chen, Jiawei Han

ChainCQG: Flow-Aware Conversational Question Generation (2021.eacl-main)

Challenge:	Existing methods only conduct network growth in a single dimension, but compound growth operators are beneficial for multiple dimensions.
Approach:	They propose a method to train BERT progressively using a Transformer model and explore alternative growth operators in each dimension via controlled comparison.
Outcome:	The proposed method speeds up BERT pre-training by 73.6% and 82.2% for the base and large models respectively while achieving comparable performances.

Copied to clipboard

Jing Gu, Mostafa Mirshekari, Zhou Yu, Aaron Sisto

Natural Language Interfaces

Challenge:	Current datasets for conversational question answering lack realistic, domain-specific training data.
Approach:	They propose a model that generates question-answer representations across dialogue turns . they use flow propagation training to improve conversational flow and fluidity .
Outcome:	The proposed model outperforms answer-aware and answer-unaware SOTA baselines significantly . it generates different types of questions with improved fluidity and coreference alignment.

HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference (2026.findings-acl)

Copied to clipboard

He Wang, Yu Gu, Fangfang Li, Zhigang Wang, Zhenghao Liu, Ning Wang, Xiaohua Li, Ge Yu

Challenge:	autoregressive inference requires repeated computation across transformer layers.
Approach:	They propose a hybrid compression framework built on both quantization and eviction . they propose varying importance metric and flexible conversion policies to reduce memory overhead .
Outcome:	The proposed framework outperforms state-of-the-art methods under memory constraints.

HAD: HAllucination Detection Language Models Based on a Comprehensive Hallucination Taxonomy (2026.acl-industry)

Copied to clipboard

Fan Xu, Xinyu Hu, Zhenghan Yu, Li Lin, Xu Zhang, Yang Zhang, Wei Zhou, Jinjie Gu, Xiaojun Wan

A Symmetric Local Search Network for Emotion-Cause Pair Extraction (2020.coling-main)

Challenge:	relying on large language models for information has raised concerns about reliability and accuracy of outputs.
Approach:	They propose a hallucination taxonomy with 11 categories for various NLG tasks and propose HAllucination Detection models which integrate hallucinism detection, span-level identification, and correction into a single inference process.
Outcome:	The proposed models outperform baselines on HaluEval, FactCHD, and FaithBench, confirming their robustness and versatility.

Copied to clipboard

Zifeng Cheng, Zhiwei Jiang, Yafeng Yin, Hua Yu, Qing Gu

Information Extraction & Text Mining

Challenge:	Existing methods for Emotion-cause pair extraction are not effective because of their lack of annotation.
Approach:	They propose a new task which aims at extracting the potential clause pairs of emotions and corresponding causes in a document.
Outcome:	The proposed method outperforms existing state-of-the-art methods on the ECPE corpus.

RankCoT: Refining Knowledge for Retrieval-Augmented Generation through Ranking Chain-of-Thoughts (2025.acl-long)

Copied to clipboard

Mingyan Wu, Zhenghao Liu, Yukun Yan, Xinze Li, Shi Yu, Zheni Zeng, Yu Gu, Ge Yu

Text Generation

Challenge:	Retrieval-Augmented Generation (RAG) models enable Large Language Models to access external knowledge.
Approach:	They propose a knowledge refinement method that incorporates reranking signals to generate CoT-based summarization based on query and retrieval documents.
Outcome:	RankCoT generates CoT-based summarization based on query and all retrieval documents . Rank CoT incorporates a self-reflection mechanism that refines the outputs .

VocalNet: Speech LLMs with Multi-Token Prediction for Faster and High-Quality Generation (2025.emnlp-main)

Copied to clipboard

Yuhao Wang, Heyang Liu, Ziyang Cheng, Ronghua Wu, Qunshan Gu, Yanfeng Wang, Yu Wang

Clues Before Answers: Generation-Enhanced Multiple-Choice QA (2022.naacl-main)

Challenge:	Experimental results show VocalNet outperforms existing open-source speech LLMs despite limited training data.
Approach:	They propose a scalable and model-agnostic training framework and a novel multi-token prediction paradigm for speech generation.
Outcome:	The proposed model outperforms open-source speech LLMs while outperforming existing open-sourced models.

Copied to clipboard

Zixian Huang, Ao Wu, Jiaying Zhou, Yu Gu, Yue Zhao, Gong Cheng

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)

Challenge:	Multiple-choice question answering (MCQA) uses text-to-text framework . but, there is an under-utilization of the decoder and knowledge that can be decoded .
Approach:	They propose a generative multiple-choice question answering model which generates a clue from the question and leverages it to enhance a reader for MCQA.
Outcome:	The proposed model outperforms text-to-text models on multiple MCQA datasets.

Copied to clipboard

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

Multilinguality

Challenge:	Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities.
Approach:	They propose a comprehensive benchmark covering 29 languages, built on an English benchmark.
Outcome:	The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark.

Dimension Reduction for Efficient Dense Retrieval via Conditional Autoencoder (2022.emnlp-main)

Copied to clipboard

Zhenghao Liu, Han Zhang, Chenyan Xiong, Zhiyuan Liu, Yu Gu, Xiaohua Li

EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)

Challenge:	Existing work reserves the principle dimensions of query and document embeddings for building more efficient retrieval systems.
Approach:	They propose to use Conditional Autoencoder to compress high-dimensional embeddings to maintain the same embeddable distribution and better recover ranking features.
Outcome:	The proposed algorithm achieves comparable ranking performance with its teacher model and makes the retrieval system more efficient.

Copied to clipboard

Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song

You Only Need One Single Token to Refine Safety Alignment (2026.findings-acl)

Challenge:	Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges .
Approach:	They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions.
Outcome:	The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset .

Copied to clipboard

Wenqian Yu, Shuo Chen, Zhijiang Li, Zhipeng Wang, Jindong Gu

ExpandR: Teaching Dense Retrievers Beyond Queries with LLM Guidance (2025.emnlp-main)

Challenge:	Excessive safety can lead to over-refusal, where models reject harmful-looking yet benign queries, severely limiting utility.
Approach:	They propose a lightweight training-based approach that reshapes the distributions of harmful and benign samples within the model’s decision space by using a single-token prefix.
Outcome:	The proposed approach can distinguish between harmful and benign samples while keeping the model frozen.

Copied to clipboard

Sijia Yao, Pengcheng Huang, Zhenghao Liu, Yu Gu, Yukun Yan, Shi Yu, Ge Yu

Information Retrieval

Challenge:	Existing methods for enhancing dense retrieval with query augmentation ignore the alignment between generation and ranking objectives.
Approach:	They propose a unified LLM-augmented dense retrieval framework that jointly optimizes both the LLM and the retriever.
Outcome:	Experimental results show that ExpandR outperforms strong baselines, achieving more than 5% improvement in retrieval performance.

Improving Domain Generalization for Prompt-Aware Essay Scoring via Disentangled Representation Learning (2023.acl-long)

Copied to clipboard

Zhiwei Jiang, Tianyi Gao, Yafeng Yin, Meng Liu, Hua Yu, Zifeng Cheng, Qing Gu

A Systematic Investigation of KB-Text Embedding Alignment at Scale (2021.acl-long)

Challenge:	Existing AES models are either prompt-specific or prompt-adaptive and cannot generalize well on “unseen” prompts.
Approach:	They propose a prompt-aware neural AES model to extract comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features.
Outcome:	The proposed model extracts comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features.

Copied to clipboard

Vardaan Pahuja, Yu Gu, Wenhu Chen, Mehdi Bahrami, Lei Liu, Wei-Peng Chen, Yu Su

Joint Optimization of Training Data and Policy in RLHF (2026.findings-acl)

Challenge:	Knowledge bases (KBs) and text often contain complementary knowledge.
Approach:	They propose a framework for aligning KB and text embeddings for joint reasoning . they also evaluate alignment methods to infuse textual information into KB embeddables .
Outcome:	The proposed framework can be used to predict link prediction on emerging entities and events using textual information.

Copied to clipboard

Zhuohao Yu, Jiali Zeng, Weizheng Gu, Mengyuan Sun, Yidong Wang, Fandong Meng, Jie Zhou, Shikun Zhang, Wei Ye

Challenge:	JODP optimizes policies on fixed training inputs, limiting the diversity of learning signals.
Approach:	They propose a framework where policy generates improved variants of training problems to enhance its own learning.
Outcome:	The proposed framework improves on safety alignment tasks by allowing 4B models to reach 8B model performance with less than 1% additional computational overhead.

Importance-based Neuron Allocation for Multilingual Neural Machine Translation (2021.acl-long)

Copied to clipboard

Wanying Xie, Yang Feng, Shuhao Gu, Dong Yu

Multilinguality

Challenge:	Existing approaches to multilingual neural machine translation tend to preserve general knowledge, but ignore language-specific knowledge.
Approach:	They propose to divide model neurons into general and language-specific parts based on their importance across languages.
Outcome:	The proposed model can preserve general knowledge but ignore language-specific knowledge on several languages, and is universal and cost-effective.

CoMMIT: Coordinated Multimodal Instruction Tuning (2025.emnlp-main)

Copied to clipboard

Xintong Li, Junda Wu, Tong Yu, Rui Wang, Yu Wang, Xiang Chen, Jiuxiang Gu, Lina Yao, Julian McAuley, Jingbo Shang

Long-Chain Reasoning Distillation via Adaptive Prefix Alignment (2026.acl-long)

Challenge:	et al., 2024) show that multimodal instruction tuning is more effective than baselines.
Approach:	They propose a multimodal balance coefficient that enables quantitative measurement of the balance of learning . they propose auxiliary regularization on the gradient to promote updating with larger step sizes .
Outcome:	The proposed method is more effective than baselines in MLLM instruction tuning.

Copied to clipboard

Zhenghao Liu, Zhuoyang Wu, Xinze Li, Yukun Yan, Shuo Wang, Zulong Chen, Yu Gu, Ge Yu, Maosong Sun

Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable reasoning capabilities, especially in solving complex mathematical problems.
Approach:	They propose a framework that exploits teacher CoTs for distillation through adaptive prefix alignment.
Outcome:	The proposed framework outperforms baseline models on multiple mathematical reasoning benchmarks by over 3%.

Copied to clipboard

Yuqi Xiong, Chunyi Peng, Zhipeng Xu, Zhenghao Liu, Zulong Chen, Yukun Yan, Shuo Wang, Yu Gu, Ge Yu

Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environments (2024.emnlp-main)

Challenge:	Existing frameworks depend on rigid, pre-defined external tools to extend perceptual capabilities of VLMs.
Approach:	They propose a framework that leverages self-emergent linguistic toolchains to enhance visual perception and reasoning.
Outcome:	The proposed framework improves the visual perception capabilities of large language models by incorporating external visual documents to address a given query.

Copied to clipboard

Yu Gu, Yiheng Shu, Hao Yu, Xiao Liu, Yuxiao Dong, Jie Tang, Jayanth Srinivasa, Hugo Latapie, Yu Su

Scaling Reasoning, Losing Control: Evaluating Instruction Following in Large Reasoning Models (2026.acl-long)

Challenge:	Large language models (LLMs) are generalist agents capable of operating within complex environments.
Approach:	They propose a class of tools that can serve as a middleware layer shielding LLMs from environmental complexity.
Outcome:	The proposed tool can shield the LLM from environmental complexity in two representative complex environments.

Copied to clipboard

Tingchen Fu, Yafu Li, Jiawei Gu, Xiaoye Qu, Yu Cheng

Large Language Models Can Be Contextual Privacy Protection Learners (2024.emnlp-main)

Challenge:	Recent advances in reasoning-oriented models have demonstrated impressive capabilities in mathematical reasoning, but their ability to adhere to user directives remains underexplored.
Approach:	They propose a benchmark to evaluate instruction-following in mathematical reasoning tasks.
Outcome:	The proposed model degrades in instruction adherence when generation length increases, but can partially recover obedience, despite increasing generation length.

Copied to clipboard

Yijia Xiao, Yiqiao Jin, Yushi Bai, Yue Wu, Xianjun Yang, Xiao Luo, Wenchao Yu, Xujiang Zhao, Yanchi Liu, Quanquan Gu, Haifeng Chen, Wei Wang, Wei Cheng

CoLLiE: Collaborative Training of Large Language Models in an Efficient Way (2023.emnlp-demo)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable linguistic comprehension and generation capability, but when applied to specialized industries, they face challenges such as hallucination, insufficient domain knowledge, and failing to incorporate the latest domain knowledge.
Approach:	They propose a paradigm for fine-tuning LLMs that effectively injects domain-specific knowledge while safeguarding inference-time data privacy.
Outcome:	The proposed model protects private data while enhancing the model's knowledge.

Copied to clipboard

Kai Lv, Shuo Zhang, Tianle Gu, Shuhao Xing, Jiawei Hong, Keyu Chen, Xiaoran Liu, Yuqing Yang, Honglin Guo, Tengxiao Liu, Yu Sun, Qipeng Guo, Hang Yan, Xipeng Qiu

Few-shot In-context Learning on Knowledge Base Question Answering (2023.acl-long)

Challenge:	Large language models (LLMs) are increasingly pivotal in a wide range of tasks . however, the resources required for training these models necessitate efficient solutions .
Approach:	They propose a library that facilitates collaborative training of large language models . they use 3D parallelism, parameter-efficient fine-tuning methods and optimizers .
Outcome:	The proposed library has proven superior training efficiency in comparison with prevalent solutions in pre-training and fine-tuning scenarios.

Copied to clipboard

Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su, Wenhu Chen

Data Annealing for Informal Language Understanding Tasks (2020.findings-emnlp)

Challenge:	KB-BINDER enables few-shot in-context learning over knowledge base questions . KBQA is a difficult problem due to the heterogeneity of knowledge bases .
Approach:	They propose a framework that enables few-shot in-context learning over KBQA tasks.
Outcome:	The proposed framework can outperform state-of-the-art models on GraphQA and MetaQA datasets.

Copied to clipboard

Jing Gu, Zhou Yu

Large Language Models Can Self-Improve (2023.emnlp-main)

Challenge:	Existing models that improve formal and informal language understanding tasks do not transfer to informal data directly.
Approach:	They propose a data annealing transfer learning procedure to bridge the performance gap on informal natural language understanding tasks.
Outcome:	The proposed procedure outperforms state-of-the-art models on three common tasks.

Copied to clipboard

Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, Jiawei Han

Low-Resource NLP

Challenge:	Large Language Models (LLMs) have excellent performance in various tasks, but fine-tuning requires extensive supervision.
Approach:	They propose to use a pre-trained Large Language Model to generate rationale-augmented answers for unlabeled questions and fine-tune the LLM using those self-generated solutions as target outputs.
Outcome:	The proposed approach improves the general reasoning ability of a 540B-parameter LLM without any ground truth label.

T2R-BENCH: A Benchmark for Real World Table-to-Report Task (2025.emnlp-main)

Copied to clipboard

Jie Zhang, Changzai Pan, Sishi Xiong, Kaiwen Wei, Yu Zhao, Xiangyu Li, Jiaxin Peng, Xiaoyan Gu, Jian Yang, Wenhan Chang, Zhenhe Wu, Jiang Zhong, Shuangyong Song, Xuelong Li

Structured Data in NLP

Challenge:	Existing table benchmarks lack the capacity to adequately assess the practical application of table reasoning in industrial applications.
Approach:	They propose a bilingual table-to-report task and a table-based benchmark to assess the quality of table reasoning.
Outcome:	The proposed task is based on a bilingual benchmark with 457 industrial tables and evaluation criteria to measure the quality of report generation.

HoWToBench: Holistic Evaluation for LLM’s Capability in Human-level Writing using Tree of Writing (2026.acl-long)

Copied to clipboard

Andrew Zhuoer Feng, Cunxiang Wang, Yu Luo, Lin Fan, Irene Zhou, Zikang Wang, Xiaotao Gu, Jie Tang, Hongning Wang, Minlie Huang

Challenge:	Evaluating the writing capabilities of large language models remains a significant challenge due to the multidimensional nature of writing skills and the limitations of existing metrics.
Approach:	They propose to model the aggregation weights of sub-features in a tree-structured workflow and propose a Chinese writing benchmark that mitigates biases.
Outcome:	The proposed tree-of-writing (ToW) measures the writing capabilities of large language models (LLMs) in Chinese and shows that it mitigates biases and achieves a 0.93 Pearson correlation with human judgments.

Towards Efficient and Effective Diffusion Language Model Inference via Semantic-Aware Adaptive Denoising (2026.acl-long)

Copied to clipboard

Fan Li, Yu Gu, Zhigang Wang, Fangling Leng, Zhenghao Liu, Ge Yu

COAST: Enhancing the Code Debugging Ability of LLMs through Communicative Agent Based Data Synthesis (2025.findings-naacl)

Challenge:	Existing acceleration works cannot accurately detect semantically stabilized tokens and then skip computation, leading to sub-optimal speedup in practice.
Approach:	They propose a semantic-aware adaptive denoising framework that encodes scalar confidence scores into an evolution-awful feature vector and clusters vectors proactively and adaptively identify semantically converged tokens.
Outcome:	The proposed framework outperforms the SOTA competitor in speed and quality . it can detect semantically stabilized tokens and skip computation, resulting in sub-optimal speedup .

Copied to clipboard

Weiqing Yang, Hanbin Wang, Zhenghao Liu, Xinze Li, Yukun Yan, Shuo Wang, Yu Gu, Minghe Yu, Zhiyuan Liu, Ge Yu

Revisiting the Roles of “Text” in Text Games (2022.findings-emnlp)

Challenge:	Existing code debugging benchmarks focus on the Code Repair stage of the code generation process.
Approach:	They propose a framework to evaluate the debugging abilities of large language models by emulating the human debug process.
Outcome:	The proposed framework outperforms human-curated and GPT-4-generated training data, enabling 7B-scale LLMs to achieve comparable debugging performance to GPT-3.5.

Copied to clipboard

Yi Gu, Shunyu Yao, Chuang Gan, Josh Tenenbaum, Mo Yu

Challenge:	Recent work has shown that random text hashes could be complementary rather than contrasting in text games.
Approach:	They propose a scheme to extract contextual information into an approximate state hash as extra input for an RNN-based text agent.
Outcome:	The proposed scheme achieves competitive performance with state-of-the-art text agents using advanced NLU techniques such as knowledge graph and passage retrieval.

Token-level Adaptive Training for Neural Machine Translation (2020.emnlp-main)

Copied to clipboard

Shuhao Gu, Jinchao Zhang, Fandong Meng, Yang Feng, Wanying Xie, Jie Zhou, Dong Yu

Multilinguality

Challenge:	Existing token imbalance phenomenon in natural language as different tokens appear with different frequencies, which leads to different learning difficulties for tokens in Neural machine translation (NMT).
Approach:	They propose to assign tokens with different frequencies to target tokens during training to encourage the model to pay more attention to low-frequency tokens.
Outcome:	The proposed model yields consistent improvements on ZH-EN, EN-RO, and EN-DE translation tasks, especially on sentences that contain more low-frequency tokens.

BeamLoRA: Beam-Constraint Low-Rank Adaptation (2025.acl-long)

Copied to clipboard

Naibin Gu, Zhenyu Zhang, Xiyu Liu, Peng Fu, Zheng Lin, Shuohuan Wang, Yu Sun, Hua Wu, Weiping Wang, Haifeng Wang

Learning Fine-Grained Grounded Citations for Attributed Large Language Models (2024.findings-acl)

Challenge:	Low-Rank Adaptation (LoRA) is one of the most efficient parameter-efficient fine-tuning methods.
Approach:	They propose to conceptualize each LoRA module as a beam where each rank corresponds to a potential sub-solution.
Outcome:	The proposed method improves performance on three base models and 12 datasets.

Copied to clipboard

Lei Huang, Xiaocheng Feng, Weitao Ma, Yuxuan Gu, Weihong Zhong, Xiachong Feng, Weijiang Yu, Weihua Peng, Duyu Tang, Dandan Tu, Bing Qin