Papers by Cheng Gao

LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)

Challenge:	Inductive reasoning is a core component of human intelligence.
Approach:	They propose a task to induce natural language rules from natural language facts using natural language as representation for knowledge instead of formal language.
Outcome:	The proposed task surpasses baselines in both automatic and human evaluations.

Copied to clipboard

Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin

Challenge:	a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities .
Approach:	They present a comparative analysis to identify and distinguish LLM activities from human activities.
Outcome:	The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities.

Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding (2023.findings-emnlp)

Copied to clipboard

Yu Zhang, Hao Cheng, Zhihong Shen, Xiaodong Liu, Ye-Yi Wang, Jianfeng Gao

Depression Detection on Social Media with Large Language Models (2025.emnlp-industry)

Challenge:	Pre-trained language models (LMs) have shown effectiveness in literature understanding tasks, especially when tuned via contrastive learning.
Approach:	They propose a multi-task contrastive learning framework that enables common knowledge sharing across different scientific literature understanding tasks while preventing task-specific skills from interfering with each other.
Outcome:	The proposed framework outperforms state-of-the-art pre-trained language models on a comprehensive dataset.

Copied to clipboard

Xiaochong Lan, Zhiguang Han, Yiming Cheng, Li Sheng, Jie Feng, Chen Gao, Yong Li

SDBench: A Survey-based Domain-specific LLM Benchmarking and Optimization Framework (2025.acl-long)

Challenge:	Existing methods for analyzing social media data lack a systematic integration of medical knowledge, causing a critical treatment gap.
Approach:	They propose a framework that leverages Large Language Models to integrate medical knowledge into social media data.
Outcome:	The proposed framework can be used to distinguish depression from transient mood changes.

Copied to clipboard

Cheng Guo, Hu Kai, Shuxian Liang, Yiyang Jiang, Yi Gao, Xian-Sheng Hua, Wei Dong

NL2Logic: AST-Guided Translation of Natural Language into First-Order Logic with Large Language Models (2026.findings-eacl)

Challenge:	acquiring domain-specific knowledge often requires professional expert manpower.
Approach:	They propose a generic framework for generating evaluation datasets for domain-specific LLMs.
Outcome:	The proposed framework reduces the reliance on expert manpower while ensuring that the collected data is uniformly distributed.

Copied to clipboard

Rizky Ramadhana Putra, Raihan Sultan Pasha Basuki, Yutong Cheng, Peng Gao

One Cognitive Loop Is Enough: SODA unlocks Pure-Text Spatial Reasoning in Large Language Models (2026.acl-long)

Challenge:	Structured reasoning approaches that parse first-order logic rules from natural language lack syntax control and semantic faithfulness.
Approach:	They propose a structured reasoning paradigm that parses first-order logic rules from natural language and delegates inference to automated solvers.
Outcome:	a proposed framework parses first-order logic rules from natural language and delegates inference to automated solvers.

Copied to clipboard

Shunwen Bai, Jiahuan Zhang, Haoran Huang, Yurun Wang, Jiale Liu, Yanxi Wu, Ningzhe Yu, Yudong Gao, Mingjun Cheng

Atoxia: Red-teaming Large Language Models with Target Toxic Answers (2025.findings-naacl)

Challenge:	Existing large language models (LLMs) lack visual input, leading to errors in basic numerical comparisons.
Approach:	They propose a spatial OODA framework that integrates the OODAC cognitive loop into multiple control tasks and integrates it into LLMs.
Outcome:	The proposed model significantly improves the spatial reasoning capabilities of large language models across multiple scenarios including SPOD-Bench, SPACE and applications.

Copied to clipboard

Yuhao Du, Zhuo Li, Pengyu Cheng, Xiang Wan, Anningzhe Gao

BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filtering (2024.emnlp-main)

Challenge:	Large language models (LLMs) are still vulnerable to generation safety vulnerabilities.
Approach:	They propose a method that A**tacks LLMs with target "toxi" given a particular harmful answer, the method generates a user query and a misleading answer opening to examine the internal defects of a given LLM.
Outcome:	The proposed method detects safety risks in open-source models and state-of-the-art models such as GPT-4o.

Copied to clipboard

Haoyu Wang, Ruirui Li, Haoming Jiang, Jinjin Tian, Zhengyang Wang, Chen Luo, Xianfeng Tang, Monica Cheng, Tuo Zhao, Jing Gao

The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020.acl-demos)

Challenge:	Retrieval-augmented Large Language Models struggle with complex inputs and noisy knowledge retrieval hindering model effectiveness.
Approach:	They propose a query generation method that integrates query generation blending with knowledge filtering to enhance retrieval-augmented LLMs.
Outcome:	The proposed approach surpasses state-of-the-art benchmarks on open-domain question answering benchmarks.

Copied to clipboard

Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (2025.naacl-long)

Challenge:	MT-DNN is an open-source natural language understanding toolkit . it allows researchers and developers to train customized deep learning models .
Approach:	They present MT-DNN, an open-source natural language understanding toolkit . it is designed to facilitate rapid customization for a broad spectrum of NLU tasks . MT supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop.
Outcome:	The proposed model can significantly compress a large model without significant performance drop.

Copied to clipboard

Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang

ReEval: Automatic Hallucination Evaluation for Retrieval-Augmented Large Language Models via Transferable Adversarial Attacks (2024.findings-naacl)

Challenge:	Existing LLMs often rely on complex prompting or extensive fine-tuning to introduce new capabilities while preserving strong generalizability.
Approach:	They propose a large-scale pre-training corpus to enhance LLM agents' capabilities . they use 103B agent-specific data encompassing 76,537 APIs .
Outcome:	The proposed training corpus outperforms open-source LLMs and commercial LLM agents on three agent benchmarks.

Copied to clipboard

Xiaodong Yu, Hao Cheng, Xiaodong Liu, Dan Roth, Jianfeng Gao

Balancing Fidelity and Plasticity: Aligning Mixed-Precision Fine-Tuning with Linguistic Hierarchies (2026.findings-acl)

Challenge:	Existing static benchmarks do not guarantee that models can use the provided evidence for answering, which is essential to avoid hallucination when the required knowledge is new or private.
Approach:	They propose to automatically perturb existing static one for dynamic evaluation by using a chatGPT framework and a set of open-domain QA datasets.
Outcome:	The proposed framework generates new test cases on two open-domain QA datasets and is human-readable and useful to trigger hallucination in LLMs.

Copied to clipboard

Changhai Zhou, Shiyang Zhang, Yuhua Zhou, Jun Gao, Qian Qiao, Shichao Weng, Weizhong Zhang, Cheng Jin

NOSE: Neural Olfactory-Semantic Embedding with Tri-Modal Orthogonal Contrastive Learning (2026.acl-long)

Challenge:	Existing quantization-aware fine-tuning methods decouple weight precision and adapter capacity, overlooking that a layer’s ability to adapt is constrained by the information preserved in its frozen weights.
Approach:	They propose a framework that jointly optimizes per-layer quantization bit-width and LoRA rank.
Outcome:	Experiments on LLaMA and Qwen models show that the proposed framework matches or approaches 16-bit baselines while using substantially less memory.

Copied to clipboard

Yanyi Su, Hongshuai Wang, Zhifeng Gao, Jun Cheng

Towards Robust Few-Shot Relation Classification: Incorporating Relation Description with Agreement (2025.findings-emnlp)

Challenge:	Existing representation methods fail to fully capture olfactory pathway . current approaches focus on isolated segments of the olefactory pathways .
Approach:	They propose a representation learning framework that aligns molecular structure, receptor sequence, and natural language description.
Outcome:	The proposed framework achieves state-of-the-art and excellent zero-shot generalization . it decouples contributions of molecular structure, receptor sequence, and natural language description .

Copied to clipboard

Mengting Hu, Jianfeng Wu, Ming Jiang, Yalan Xie, Zhunheng Wang, Rui Ying, Xiaoyi Liu, Ruixuan Xu, Hang Gao, Renhong Cheng

SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)

Challenge:	Existing approaches to recognize relational relationships with a few support samples are limited for unlimited queries.
Approach:	They propose a simple but effective framework that uses relation descriptions as external knowledge to enhance the model’s comprehension of the relation semantics.
Outcome:	The proposed framework outperforms strong baselines while being robust against various NOTA rates.

Copied to clipboard

Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke

Challenge:	Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis.
Approach:	They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis.
Outcome:	SciAssess evaluates 11 LLMs on multiple tasks across scientific fields.

Efficient Sequence Learning with Group Recurrent Networks (N18-1)

Copied to clipboard

Fei Gao, Lijun Wu, Li Zhao, Tao Qin, Xueqi Cheng, Tie-Yan Liu

Green & Sustainable NLP

Challenge:	Recurrent neural networks have achieved state-of-the-art results in many artificial intelligence tasks, such as language modeling, neural machine translation and speech recognition.
Approach:	They propose an efficient architecture to improve the efficiency of such RNN model training by adopting the group strategy for recurrent layers while exploiting the representation rearrangement strategy between layers as well as time steps.
Outcome:	The proposed architecture achieves comparable or better accuracy compared with baselines, with a much smaller number of parameters and at a lower computational cost.

ALiiCE: Evaluating Positional Fine-grained Citation Generation (2025.naacl-long)

Copied to clipboard

Yilong Xu, Jinhua Gao, Xiaoming Yu, Baolong Bi, Huawei Shen, Xueqi Cheng

Challenge:	Existing research on citation generation is limited to sentence-level statements . positional fine-grained citations can appear anywhere within sentences .
Approach:	They propose a framework that allows LLMs to generate citations from sentences . they use dependency tree-based methods to parse sentence-level claims into atomic claims .
Outcome:	The proposed framework evaluates citation quality using three metrics including positional fine-grained citation recall, precision, and coefficient of variation of citation positions.

Document Segmentation Matters for Retrieval-Augmented Generation (2025.findings-acl)

Copied to clipboard

Zhitong Wang, Cheng Gao, Chaojun Xiao, Yufei Huang, Shuzheng Si, Kangyang Luo, Yuzhuo Bai, Wenhao Li, Tangjian Duan, Chuancheng Lv, Guoshan Lu, Gang Chen, Fanchao Qi, Maosong Sun

Syntactic Text Processing

Challenge:	Existing rule-based chunking methods lead to suboptimal splits, where overly large chunks introduce irrelevant information and small chunks lack semantic coherence.
Approach:	They propose a method that leverages document summaries as pseudo-instructions to guide chunking by computing semantic similarity between sentences and the summary.
Outcome:	Experiments on multiple open-domain question-answering benchmarks show that PIC significantly improves retrieval accuracy (Hits@k) and end-to-end QA performance (Exact Match) without any additional training.

A Survey of Post-Training Scaling in Large Language Models (2025.acl-long)

Copied to clipboard

Hanyu Lai, Xiao Liu, Junjie Gao, Jiale Cheng, Zehan Qi, Yifan Xu, Shuntian Yao, Dan Zhang, Jinhua Du, Zhenyu Hou, Xin Lv, Minlie Huang, Yuxiao Dong, Jie Tang

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities (2025.naacl-long)

Challenge:	Large language models (LLMs) have demonstrated proficiency in understanding and generating human natural languages.
Approach:	They propose a framework for scaling large language models using supervised fine-tuning, RLxF and test-time compute methodologies.
Outcome:	The proposed model can be used to understand and generate human natural languages.

Copied to clipboard

Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, Jianfeng Gao

Robustness in NLP

Challenge:	Recent research shows that Large Language Models (LLMs) are vulnerable to automated jailbreak attacks.
Approach:	They propose a framework that crafts adversarial LLMs with enhanced jailbreak ability.
Outcome:	ADV-LLM significantly reduces the computational cost of generating adversarial suffixes while achieving nearly 100% ASR on various open-source LLMs.

Simple but Effective Compound Geometric Operations for Temporal Knowledge Graph Completion (2024.acl-long)

Copied to clipboard

Rui Ying, Mengting Hu, Jianfeng Wu, Yalan Xie, Xiaoyi Liu, Zhunheng Wang, Ming Jiang, Hang Gao, Linlin Zhang, Renhong Cheng

Task-Aware Specialization for Efficient and Robust Dense Retrieval for Open-Domain Question Answering (2023.acl-short)

Challenge:	Current methods embed factual knowledge into continuous vector space and apply geometric operations to learn potential patterns in temporal knowledge graphs.
Approach:	They propose a temporal knowledge graph completion method that uses two geometric operations to learn missing facts in temporal graphs.
Outcome:	The proposed method significantly outperforms existing temporal knowledge graph embedding models.

Copied to clipboard

Hao Cheng, Hao Fang, Xiaodong Liu, Jianfeng Gao

TRUST: Towards Robust Social Bot Detection via Uncertainty-Guided Pseudo-Labeling and Graph Structure Purification (2026.findings-acl)

Challenge:	Existing dense retrieval models are parameter-inefficient and underperform sparse counterparts.
Approach:	They propose a task-aware specialization for dEnse Retrieval architecture that enables parameter sharing by interleaving shared and specialized blocks in a single encoder.
Outcome:	The proposed architecture surpasses BM25 on questions and passages using 60% of the parameters as bi-encoder dense retrievers.

Copied to clipboard

Ruixuan Xu, Mengting Hu, Zhunheng Wang, Ming Jiang, Rui Ying, Zhen Zhang, Hang Gao, Shuaipeng Liu, Renhong Cheng

Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Other (2024.findings-naacl)

Challenge:	Existing graph-based detection models are vulnerable to deceptive message propagation, where bots deliberately interact with legitimate users.
Approach:	They propose a framework to mitigate deceptive message propagation by node-level uncertainty estimation and graph structure purification.
Outcome:	The proposed framework improves on three benchmark datasets and six GNN backbones on real-world social bots.

Copied to clipboard

Yifei Gao, Jie Ou, Lei Wang, Yuting Xiao, Xiangzhiyuan Xiangzhiyuan, Ruiting Dai, Jun Cheng

Training a Utility-based Retriever Through Shared Context Attribution for Retrieval-Augmented Language Models (2025.emnlp-main)

Challenge:	Emergent Large Language Models (LLMs) use extraordinary performance and powerful deduction capacity to discern from traditional language models.
Approach:	They propose a method that uses weights to compensate quantization error and learnable singular value incremental (LSI) LSI is a technique that helps weights compensate each other conditioned on activation.
Outcome:	The proposed method achieves state-of-the-art performance in diverse quantization settings, no matter in weight-only, weight-activation or extremely low bit scenarios.

Copied to clipboard

Yilong Xu, Jinhua Gao, Xiaoming Yu, Yuanhai Xue, Baolong Bi, Huawei Shen, Xueqi Cheng

AutoPKG: An Automated Framework for Dynamic E-commerce Product-Attribute Knowledge Graph Construction (2026.findings-acl)

Challenge:	Utility-based retrieval has emerged as a promising topic for downstream tasks . however, capturing passage utility accurately remains unexplored due to insufficient understanding .
Approach:	They propose a framework for training utility-based retrievers in Retrieval-Augmented Language Models . it incorporates multi-task generalization and inter-passage interaction to improve performance .
Outcome:	The proposed framework improves performance on ten datasets across different tasks.

Copied to clipboard

Pollawat Hongwimol, Haoning Shang, Chutong Wang, Zhichao Wan, Yi Gao, Yuanming Li, Lin Gui, Wenhao Sun, Cheng Yu

TestAgent: An Adaptive and Intelligent Expert for Human Assessment (2025.findings-acl)

Challenge:	Product attribute extraction in e-commerce is bottlenecked by ontologies that are inconsistent, incomplete, and costly to maintain.
Approach:	They propose a multi-agent Large Language Model framework that constructs a Product-attribute Knowledge Graph from multimodal product content.
Outcome:	The proposed framework achieves 0.953 WKE for product types, 0.724 WKEs for attribute keys, and 0.531 edge-level accuracy for value assertions after canonicalization on a large real-world marketplace catalog dataset from Lazada (Alibaba).

Copied to clipboard

Junhao Yu, Yan Zhuang, Yuxuan Sun, Weibo Gao, Qi Liu, Mingyue Cheng, Zhenya Huang, Enhong Chen

ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models (2025.acl-long)

Challenge:	Existing adaptive testing methods face several challenges due to mechanized nature of most algorithms and noisy response data.
Approach:	They propose to use large language models to enhance adaptive testing through interactive engagement to capture test-takers’ responses and anomalies.
Outcome:	The proposed agent achieves more accurate results with 20% fewer questions than state-of-the-art baselines and testers preferred it in speed, smoothness, and other dimensions.

Copied to clipboard

Hanxing Ding, Shuchang Tao, Liang Pang, Zihao Wei, Jinyang Gao, Bolin Ding, Huawei Shen, Xueqi Cheng

Reinforcement Learning on Pre-Training Data (2026.acl-long)

Challenge:	Existing approaches to tool learning rely on hand-crafted prompts and natural language reasoning, making multi-step planning difficult and lacking precise error diagnosis and reflection mechanisms.
Approach:	They propose a framework that reformulates tool learning as a code generation task.
Outcome:	The proposed framework achieves superior performance in task completion accuracy and execution reliability compared to existing approaches.

Copied to clipboard

Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang

MEIC-DT: Memory-Efficient Incremental Clustering for Long-Text Coreference Resolution with Dual-Threshold Constraints (2026.findings-acl)

Challenge:	Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings.
Approach:	They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data.
Outcome:	Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base.

Copied to clipboard

Kangyang Luo, Shuzheng Si, Yuzhuo Bai, Cheng Gao, Zhitong Wang, Cheng Huang, Yingli Shen, Yufeng Han, Wenhao Li, Cunliang Kong, Maosong Sun

Information Extraction & Text Mining

Challenge:	Existing supervised neural methods are underexplored for coreference resolution, especially in incremental clustering.
Approach:	They propose a dual-threshold incremental clustering approach based on a lightweight Transformer.
Outcome:	Experiments on common benchmarks show that MEIC-DT achieves highly competitive coreference performance under stringent memory constraints.

Multi-Label Few-Shot Learning for Aspect Category Detection (2021.acl-long)

Copied to clipboard

Mengting Hu, Shiwan Zhao, Honglei Guo, Chao Xue, Hang Gao, Tiegang Gao, Renhong Cheng, Zhong Su

High-Order Semantic Alignment for Unsupervised Fine-Grained Image-Text Retrieval (2024.lrec-main)

Challenge:	Existing few-shot learning methods focus on single-label predictions, which can not work well for ACD since a sentence may contain multiple aspect categories.
Approach:	They propose a few-shot learning method that uses the prototypical network to learn aspects from a set of aspects.
Outcome:	The proposed method significantly outperforms baseline methods on three datasets.

Copied to clipboard

Rui Gao, Miaomiao Cheng, Xu Han, Wei Song

CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision (2025.acl-long)

Challenge:	Existing studies focus on learning global or local correspondence, but lack fine-grained local-global alignment.
Approach:	They propose a High Order Semantic Alignment (HOSA) model that can provide complementary and comprehensive semantic clues to infer correlation scores.
Outcome:	The proposed model outperforms state-of-the-art models in retrieving the most relevant results.

Copied to clipboard

YifeiLu YifeiLu, Fanghua Ye, Jian Li, Qiang Gao, Cheng Liu, Haibo Luo, Nan Du, Xiaolong Li, Feiliang Ren

Programming Languages in NLP

Challenge:	Existing approaches to tool invocation are often unnecessarily long and require lengthy reasoning paths.
Approach:	They propose a framework for stepwise code generation that improves LLM tool invocation . they incorporate two distinct process rewards: the On-the-spot and the Latent Reward .
Outcome:	The proposed framework improves LLM tool invocation by leveraging the concise nature of code.

Dialogue Summarization with Static-Dynamic Structure Fusion Graph (2023.acl-long)

Copied to clipboard

Shen Gao, Xin Cheng, Mingzhe Li, Xiuying Chen, Jinpeng Li, Dongyan Zhao, Rui Yan

GR1: Reinforcement-Enhanced LLM for Geoscience Reasoning (2026.findings-acl)

Challenge:	Dialogue summarization is a challenging task since it has dynamic interaction nature and inconsistent information flow among various speakers.
Approach:	They propose a Static-Dynamic graph-based Dialogue Summarization model which fuses prior knowledge from human expertise and adaptively learns the graph structure in an end-to-end learning fashion.
Outcome:	The proposed model can help people capture the highlights of a semi-structured and multi-participant dialogue without reviewing the complex dialogue context.

Copied to clipboard

Yule Xie, Jiaxin Ding, Cheng Deng, Shiqing Gao, Junran Zhang, Sibo Zhang, Zeyuan Wang, Ke Wu, Xin Ding, Luoyi Fu, Meng Jin, Xinbing Wang

Knowledge-Rich Self-Supervision for Biomedical Entity Linking (2022.findings-emnlp)

Challenge:	Recent advances in large language models have demonstrated RL's substantial capacity to enhance multi-step reasoning beyond what supervised instruction tuning achieves.
Approach:	They propose a framework that converts multimodal questions into descriptive text . they propose RL-enhanced geoscience reasoning that can be fine-tuned to a text-only level .
Outcome:	The proposed framework improves accuracy and accuracy on multimodal questions while preserving answerability and difficulty.

Copied to clipboard

Sheng Zhang, Hao Cheng, Shikhar Vashishth, Cliff Wong, Jinfeng Xiao, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, Hoifung Poon

GLTW: Joint Improved Graph Transformer and LLM via Three-Word Language for Knowledge Graph Completion (2025.findings-acl)

Challenge:	Entity linking is challenging in high-value domains with myriad entities . standard classification approaches suffer from the annotation bottleneck .
Approach:	They propose a self-supervised approach to learn domain knowledge for biomedical entity linking . it generates self-reported mention examples on unlabeled text and trains contextual encoder .
Outcome:	The proposed method outperforms existing methods by 20 points in accuracy on biomedical datasets.

Copied to clipboard

Kangyang Luo, Yuzhuo Bai, Cheng Gao, Shuzheng Si, Zhu Liu, Yingli Shen, Zhitong Wang, Cunliang Kong, Wenhao Li, Yufei Huang, Ye Tian, Xuantang Xiong, Lei Han, Maosong Sun

RoseLoRA: Row and Column-wise Sparse Low-rank Adaptation of Pre-trained Language Model for Knowledge Editing and Fine-tuning (2024.emnlp-main)

Challenge:	Existing knowledge graphs lack the ability to integrate structural information into LLMs and output predictions deterministically.
Approach:	They propose a method which encodes structural information of KGs and merges it with LLMs to enhance KGC performance.
Outcome:	The proposed method improves the performance of KG Completion datasets on KGs by integrating structural information with LLMs.

Copied to clipboard

Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, Jing Gao

Chain-of-Skills: A Configurable Model for Open-Domain Question Answering (2023.acl-long)

Challenge:	Pre-trained language models have strong generalizability, but fine-tuning involves updating all parameters, rendering full fine-uning resource-intensive.
Approach:	They propose a parameter-efficient fine-tuning method that updates all pre-trained parameters during inference.
Outcome:	The proposed method outperforms baseline methods on five benchmarks across 20 datasets.

Copied to clipboard

Kaixin Ma, Hao Cheng, Yu Zhang, Xiaodong Liu, Eric Nyberg, Jianfeng Gao

Information Retrieval

Challenge:	Using customized retrieval models, model transferability and scalability are limited.
Approach:	They propose a modular retrieval model where individual modules correspond to key skills that can be reused across datasets.
Outcome:	The proposed model outperforms self-supervised retrievers in zero-shot evaluations and achieves state-of-the-art fine-tuned retrieval performance on NQ, HotpotQA and OTT-QA.

Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps (2025.findings-acl)

Copied to clipboard

Jiashun Cheng, Aochuan Chen, Nuo Chen, Ziqi Gao, Yuhan Li, Jia Li, Fugee Tsung

The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)

Challenge:	Low-Rank Adaptation (LoRA) has emerged as a prominent technique for fine-tuning large foundation models.
Approach:	They propose a low-rank Adaptation technique that harnesses the expressiveness of spectral bases to re-parameterize LoRA from a sparse spectral subspace.
Outcome:	The proposed technique achieves greater efficiency with fewer parameters than baselines on various downstream tasks, including commonsense reasoning, math reasoning, and code generation.

Copied to clipboard

Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun

ImCoref-CeS: An Improved Lightweight Pipeline for Coreference Resolution with LLM-based Checker-Splitter Refinement (2026.acl-long)

Challenge:	Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations.
Approach:	They propose a training data arrangement framework that allows for continual learning and loss reduction.
Outcome:	The proposed framework promotes continual learning and loss reduction on unseen tasks.

Copied to clipboard

Kangyang Luo, Yuzhuo Bai, Shuzheng Si, Cheng Gao, Zhitong Wang, Yingli Shen, Wenhao Li, Zhu Liu, Yufeng Han, Jiayi Wu, Cunliang Kong, Maosong Sun

Information Extraction & Text Mining

Challenge:	Existing supervised neural methods for coreference resolution are underexplored . current methods rely on small language models, but their potential is underexploited .
Approach:	They propose a framework that integrates an enhanced supervised model with LLM-based reasoning.
Outcome:	The proposed method surpasses existing state-of-the-art methods in coreference resolution.

HiTab: A Hierarchical Table Dataset for Question Answering and Natural Language Generation (2022.acl-long)

Copied to clipboard

Zhoujun Cheng, Haoyu Dong, Zhiruo Wang, Ran Jia, Jiaqi Guo, Yan Gao, Shi Han, Jian-Guang Lou, Dongmei Zhang

Natural Language Interfaces

Challenge:	Existing studies on table reasoning focus on flat tables and hierarchical tables . a new dataset, HiTab, aims to examine numerical reasoning over hierarchic tables based on hierarchically structured tables - a strong challenge for existing baselines and a valuable benchmark for future research.
Approach:	They propose a hierarchical question answering and natural language generation dataset to study hierarchic tables.
Outcome:	The proposed model shows that it is effective in QA and natural language generation over hierarchical tables.

UnitedQA: A Hybrid Approach for Open Domain Question Answering (2021.acl-long)

Copied to clipboard

Hao Cheng, Yelong Shen, Xiaodong Liu, Pengcheng He, Weizhu Chen, Jianfeng Gao

Natural Language Interfaces

Challenge:	Recent work on open-domain question answering focuses on either extractive or generative readers exclusively.
Approach:	They propose a hybrid approach to extractive and generative readers that leverages both models.
Outcome:	The proposed approach outperforms state-of-the-art models on NaturalQuestions and TriviaQA respectively.

Is MultiWOZ a Solved Task? An Interactive TOD Evaluation Framework with User Simulator (2022.findings-emnlp)

Copied to clipboard

Qinyuan Cheng, Linyang Li, Guofeng Quan, Feng Gao, Xiaofeng Mou, Xipeng Qiu

Dialogue Systems & Conversational Agents

Challenge:	Task-oriented dialogue systems are drawing more attention in recent studies . current evaluation methods use annotated utterances in multi-turn dialogue sessions .
Approach:	They propose an interactive evaluation framework for task-oriented dialogue (TOD) they build a goal-oriented user simulator based on pre-trained models and use it to generate dialogues .
Outcome:	The proposed evaluation framework achieves 98% inform and success rates in the multi-woz dataset and the proposed scores measure the response quality besides the inform and successful rates.

Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog (P19-1)

Copied to clipboard

Zhe Gan, Yu Cheng, Ahmed Kholy, Linjie Li, Jingjing Liu, Jianfeng Gao

Natural Language Interfaces

Challenge:	Existing models for visual dialog infer the answer through multiple reasoning steps.
Approach:	They propose a model for visual dialog that uses multi-step reasoning to answer questions about an image.
Outcome:	The proposed model achieves a new state-of-the-art of 64.47% on the VisDial v1.0 dataset .

Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge (2022.findings-emnlp)

Copied to clipboard

Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao

LogosKG: Hardware-Optimized Scalable and Interpretable Knowledge Graph Retrieval (2026.acl-long)

Challenge:	Existing open-domain question answering methods rely on the retriever to gather all evidence in isolation, but our approach uses an intermediary module to perform a chain of reasoning over the retrieved set.
Approach:	They propose a new open-domain question answering framework that integrates an intermediary module into the current retriever-reader pipeline and integrates it into the model.
Outcome:	The proposed framework outperforms the state-of-the-art on two OTT-QA datasets with an exact match score of 47.3 (45% relative gain).

Copied to clipboard

He Cheng, Yifu Wu, Saksham Khatwani, Maya Kruse, Dmitriy Dligach, Timothy A. Miller, Majid Afshar, Yanjun Gao

Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering (2025.acl-long)

Challenge:	Existing systems struggle to balance efficiency, scalability, and interpretability.
Approach:	They propose a hardware-aligned framework that enables scalable and interpretable k-hop retrieval on large KGs.
Outcome:	The proposed framework scales to billion-edge graphs without loss of retrieval fidelity.

Copied to clipboard

Shuzheng Si, Haozhe Zhao, Gang Chen, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Kaikai An, Kangyang Luo, Chen Qian, Fanchao Qi, Baobao Chang, Maosong Sun

Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs (2024.emnlp-main)

Challenge:	Existing studies show that training LLMs on data containing unfamiliar knowledge during instruction tuning can encourage hallucinations.
Approach:	They propose a framework that measures how familiar the LLM is with instruction data and introduce an expert-aligned reward model to ensure the quality of selected samples.
Outcome:	The proposed framework reduces hallucinations while maintaining a competitive ability to follow instructions.

Copied to clipboard

Cheng Gao, Chaojun Xiao, Zhenghao Liu, Huimin Chen, Zhiyuan Liu, Maosong Sun

Information Retrieval

Challenge:	Existing studies focus on case-to-case retrieval using lengthy queries, which does not match real-world scenarios.
Approach:	They propose a method to construct query-candidate pairs and build the largest LCR dataset to date, LEAD.
Outcome:	Experimental results show that the method can provide ample training signals for LCR models.

TLUE: A Tibetan Language Understanding Evaluation Benchmark (2025.emnlp-main)

Copied to clipboard

Fan Gao, Cheng Huang, Yutong Liu, Nyima Tashi, Xiangxiang Wang, Thupten Tsering, Ban Ma-bao, Renzeng Duojie, Gadeng Luosang, Rinchen Dongrub, Dorje Tashi, Xiao Feng Cd, Yongbin Yu, Hao Wang

GAST: Gradient-aligned Sparse Tuning of Large Language Models with Data-layer Selection (2026.eacl-long)

Challenge:	Low-resource languages, like Tibetan, remain underrepresented in large language models' evaluations.
Approach:	They propose a Tibetan Language Understanding Evaluation Benchmark to assess LLMs' proficiency in Tibetan . they use a multi-task understanding benchmark and a safety benchmark to evaluate models .
Outcome:	The proposed benchmark shows that most large language models perform below the random baseline, especially in Tibetan language processing.

Copied to clipboard

Kai Yao, Zhenghan Song, Kaixin Wu, Mingjie Zhong, Danzhao Cheng, Zhaorui Tan, Yixin Ji, Penglei Gao

Improving Domain Generalization for Prompt-Aware Essay Scoring via Disentangled Representation Learning (2023.acl-long)

Challenge:	Existing methods focus on layer-selective and data-selectory fine-tuning, but ignore the fact that different data points contribute varying degrees to distinct model layers.
Approach:	They propose a method that performs selective fine-tuning at both data and layer dimensions as integral components of a unified optimization strategy.
Outcome:	Experiments show that the proposed method outperforms baseline methods in terms of performance and performance.

Copied to clipboard

Zhiwei Jiang, Tianyi Gao, Yafeng Yin, Meng Liu, Hua Yu, Zifeng Cheng, Qing Gu

Finch: Benchmarking Finance & Accounting across Spreadsheet-Centric Enterprise Workflows (2026.findings-acl)

Challenge:	Existing AES models are either prompt-specific or prompt-adaptive and cannot generalize well on “unseen” prompts.
Approach:	They propose a prompt-aware neural AES model to extract comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features.
Outcome:	The proposed model extracts comprehensive representation for essay scoring, including both prompt-invariant and prompt-specific features.

Copied to clipboard

Haoyu Dong, Pengkun Zhang, Yan Gao, Xuanyu Dong, Yilin Cheng, Mingzhe Lu, Adina Yakefu, Shuxin Zheng

IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory (2025.acl-long)

Challenge:	FinWorkBench evaluates real-world enterprise-grade finance and accounting workflows . a human evaluation of GPT 5.1 Pro passes only 38.4% of workflows, a study finds .
Approach:	They propose a workflow construction process that combines LLM-assisted mining and expert annotation to build 172 composite workflows.
Outcome:	The proposed process combines expert annotation with LLM-assisted mining of workflows from authentic enterprise environments.

Copied to clipboard

Wei Song, Zhenya Huang, Cheng Cheng, Weibo Gao, Bihan Xu, GuanHao Zhao, Fei Wang, Runze Wu

Linguistic Theories

Challenge:	Large language models have demonstrated exceptional performance across a wide range of tasks . however, selecting the optimal LLM to respond to a user query often necessitates a delicate balance between performance and cost.
Approach:	They propose a multi-LLM routing framework that efficiently routes user queries to the most suitable LLM.
Outcome:	The proposed framework outperforms baseline methods in terms of effectiveness and interpretability.

Adaptive Token Biaser: Knowledge Editing via Biasing Key Entities (2024.findings-emnlp)

Copied to clipboard

Baolong Bi, Shenghua Liu, Yiwei Wang, Lingrui Mei, Hongcheng Gao, Yilong Xu, Xueqi Cheng

Challenge:	Existing methods to update parametric knowledge of large language models (LLMs) are outdated and incontext editing (KE) is not effective due to the substantial cost associated with retraining.
Approach:	They propose a new decoding technique that enhances in-context editing (ICE) they propose to use parametric knowledge to update the models' knowledge .
Outcome:	The proposed technique improves ICE performance while incurring only half the latency.

OpenCoder: The Open Cookbook for Top-Tier Code Large Language Models (2025.acl-long)

Copied to clipboard

Siming Huang, Tianhao Cheng, Jason Klein Liu, Weidi Xu, Jiaran Hao, Liuyihan Song, Yang Xu, Jian Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Xianzhen Luo, Qiufeng Wang, YuanTao Fan, Qingfu Zhu, Zhaoxiang Zhang, Yang Gao, Jie Fu, Qian Liu, Houyi Li, Ge Zhang, Yuan Qi, Xu Yinghui, Wei Chu, Zili Wang

Posterior Differential Regularization with f-divergence for Improving Model Robustness (2021.naacl-main)

Challenge:	Code LLMs lack reproducible data pipelines and training protocols for reproducible advancements in code intelligence.
Approach:	They propose a top-tier code LLM that releases model weights and inference code . reproducible data pipelines, rigorous experimental ablation results and training protocols are included .
Outcome:	The proposed model achieves comparable performance to leading models and serves as an "open cookbook" reproducible training data, rigorous experimental ablation results, and detailed training protocols are also included in the model.

Copied to clipboard

Hao Cheng, Xiaodong Liu, Lis Pereira, Yaoliang Yu, Jianfeng Gao

Robustness in NLP

Challenge:	Recent studies show that pre-trained models suffer catastrophic degradation in out-of-domain generalization to datasets with domain shift or adversarial scenarios.
Approach:	They propose to regularize the posterior difference between clean and noisy inputs by using a Jacobian regularization framework and a virtual adversarial training framework.
Outcome:	The proposed framework can improve model robustness in fully supervised and semi-supervised settings.

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models (2025.acl-long)

Copied to clipboard

Xinxin Liu, Aaron Thomas, Cheng Zhang, Jianyi Cheng, Yiren Zhao, Xitong Gao

Soft Contextual Data Augmentation for Neural Machine Translation (P19-1)

Challenge:	Existing methods for fine-tuning large language models require expensive training on consumer-grade hardwares.
Approach:	They propose a sparsity-based approach that introduces trainable sparse adaptations to the weight matrices in the model and offers greater flexibility in selecting fine-tuned parameters.
Outcome:	The proposed method outperforms other methods for a simple yet effective baseline for nLP tasks while sacrificing performance.

Copied to clipboard

Fei Gao, Jinhua Zhu, Lijun Wu, Yingce Xia, Tao Qin, Xueqi Cheng, Wengang Zhou, Tie-Yan Liu

Machine Translation

Challenge:	Existing methods for enhancing training data are limited in natural language tasks due to text characteristics.
Approach:	They propose a data augmentation method that softly augments a randomly chosen word in a sentence by its contextual mixture of multiple related words.
Outcome:	The proposed method outperforms baseline methods on small and large scale machine translation datasets.

DocHieNet: A Large and Diverse Dataset for Document Hierarchy Parsing (2024.emnlp-main)

Copied to clipboard

Hangdi Xing, Changxu Cheng, Feiyu Gao, Zirui Shao, Zhi Yu, Jiajun Bu, Qi Zheng, Cong Yao

Challenge:	Existing methods for document hierarchy parsing are limited due to the small scale and inconsistency of datasets.
Approach:	They propose a document hierarchy parsing dataset to compensate for the data scarcity problem and propose 'dHP' framework to grasp fine-grained text content and coarse-grounded pattern at layout element level.
Outcome:	The proposed framework grasps both fine-grained text content and coarse-grounded pattern at layout element level, enhancing the capacity of pre-trained text-layout models in handling multi-page and multi-level challenges.

ProtoCycle: Reflective Tool-Augmented Planning for Text-Guided Protein Design (2026.findings-acl)

Copied to clipboard

Yutang Ge, Guojiang Zhao, Sihang Li, Zheng Cheng, Zifeng Zhao, Hanchen Xia, Guolin Ke, Linfeng Zhang, Zhifeng Gao, Yu Guang Wang

Challenge:	Recent deep generative models have already shown encouraging * Equal contribution.
Approach:	They propose to use generic instruction-tuned LLMs as direct text-to-sequence generators to achieve this goal.
Outcome:	Recent studies show that reflection improves sequence quality and alignment while maintaining competitive foldability.

Open Domain Question Answering with A Unified Knowledge Interface (2022.acl-long)

Copied to clipboard

Kaixin Ma, Hao Cheng, Xiaodong Liu, Eric Nyberg, Jianfeng Gao

Structured Data in NLP

Challenge:	a retriever-reader framework is popular for open domain question answering . however, accessing heterogeneous knowledge sources through a unified interface remains unknown .
Approach:	They propose a retriever-reader framework that uses explicit knowledge to access heterogeneous knowledge sources through a unified interface.
Outcome:	The proposed framework can benefit from the expanded knowledge index, the authors show . their approach sets the single-model state-of-the-art on Natural Questions .

Nirvana: A Specialized Generalist Model With Task-Aware Memory Mechanism (2026.acl-long)

Copied to clipboard

Yuhua Jiang, Shuang Cheng, Yihao Liu, Ermo Hua, Che Jiang, Weigao Sun, Yu Cheng, Feifei Gao, Biqing Qi, Bowen Zhou

Targeted Adversarial Training for Natural Language Understanding (2021.naacl-main)

Challenge:	Existing models lack task-guided specialized memory mechanisms . specialized generalist models excel at general language tasks but struggle in specialized domains.
Approach:	They propose a specialized generalist model with specialized memory and updater that can optimize for specialized domains.
Outcome:	The proposed model matches or surpasses baselines on general benchmarks and achieves lowest perplexity across specialized domains.

Copied to clipboard

Lis Pereira, Xiaodong Liu, Hao Cheng, Hoifung Poon, Jianfeng Gao, Ichiro Kobayashi

Robustness in NLP

Challenge:	Existing adversarial training approaches focus on making adversarials less expensive or regularizing rather than replacing the standard training objective.
Approach:	They propose an algorithm to introspect current mistakes and prioritize adversarial training steps to where the model errs the most.
Outcome:	The proposed algorithm improves adversarial training for natural language understanding by introspecting mistakes and prioritizing training steps to where the model errs the most.

Bridging Kernel Drivers and Virtual Device Models with LLM-Powered Automation (2026.acl-demo)

Copied to clipboard

Mingyu Wang, Bin Yu, Wenjian Lu, Zhi Wang, Gao Kefeng, Cheng Wen, Xu Lu, Cong Tian

Diversifying the Expert Knowledge for Task-Agnostic Pruning in Sparse Mixture-of-Experts (2025.findings-acl)

Challenge:	Linux kernel device drivers are tightly coupled with hardware, making them difficult to execute and test without physical devices.
Approach:	They present a tool that generates QEMU-based virtual devices directly from Linux driver source code.
Outcome:	The proposed tool generates QEMU-based virtual devices directly from Linux driver source code.

Copied to clipboard

Zeliang Zhang, Xiaodong Liu, Hao Cheng, Chenliang Xu, Jianfeng Gao

Challenge:	Large Language Models (LLMs) have outstanding performance by learning a large number of model parameters on large amounts of data.
Approach:	They propose a method of grouping and pruning similar experts to improve the model’s parameter efficiency by a range of natural language tasks.
Outcome:	The proposed method outperforms other model pruning methods on a range of natural language tasks.

BvSP: Broad-view Soft Prompting for Few-Shot Aspect Sentiment Quad Prediction (2024.acl-long)

Copied to clipboard

Yinhao Bai, Yalan Xie, Xiaoyi Liu, Yuhua Zhao, Zhixin Han, Mengting Hu, Hang Gao, Renhong Cheng

Sentiment Analysis

Challenge:	Aspect sentiment quad prediction aims to predict aspects due to distinct data distribution.
Approach:	They propose a method that aggregates multiple templates with a broader view . they first construct a few-shot ASQP dataset that contains richer categories .
Outcome:	The proposed method outperforms the state-of-the-art methods under four few-shot settings and other public datasets.

ECoK: Emotional Commonsense Knowledge Graph for Mining Emotional Gold (2024.findings-acl)

Copied to clipboard

Zhunheng Wang, Xiaoyi Liu, Mengting Hu, Rui Ying, Ming Jiang, Jianfeng Wu, Yalan Xie, Hang Gao, Renhong Cheng

Parameter-Efficient Fine-Tuning via Circular Convolution (2025.findings-acl)

Challenge:	Existing knowledge graphs focus on the representation and reasoning of general factual knowledge, while there are significant deficiencies in the understanding and reasoning for emotional knowledge.
Approach:	They propose a commonsense knowledge graph that can be used to represent emotional knowledge by combining theories from psychology, cognitive science, and linguistics.
Outcome:	The proposed model surpasses GPT-4-Turbo in the emotion-related tasks.

Copied to clipboard

Aochuan Chen, Jiashun Cheng, Zijing Liu, Ziqi Gao, Fugee Tsung, Yu Li, Jia Li