Papers by Nan Liu

FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)

Challenge:	Statistical language modeling and translation with transformers have found many successful applications in program understanding and generation tasks.
Approach:	They propose an architecture-independent approach for leveraging syntactic hierarchies of source code . they use syntax trees to extract syntak hierarchical structures and integrate them into context window .
Outcome:	The proposed approach achieves state-of-the-art in code completion and summarization for Python in the CodeXGLUE benchmark.

Copied to clipboard

Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev

Reasoning

Challenge:	Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity.
Approach:	They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models.
Outcome:	The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models.

Dual-Channel Evidence Fusion for Fact Verification over Texts and Tables (2022.naacl-main)

Copied to clipboard

Nan Hu, Zirui Wu, Yuxuan Lai, Xiao Liu, Yansong Feng

Fact & Claim Verification

Challenge:	Existing fact extraction and verification tasks only consider evidence of a single format . Existing models convert evidence into either sentences or tables, thus losing context information .
Approach:	They propose a Dual Channel Unified Format fact verification model which unifies various evidence into parallel streams, i.e., natural language sentences and a global evidence table, simultaneously.
Outcome:	The proposed model outperforms existing models in two formats by a large margin . it makes the most of existing tables and tables to absorb evidence of two formats .

RAGEval: Scenario Specific RAG Evaluation Dataset Generation Framework (2025.acl-long)

Copied to clipboard

Kunlun Zhu, Yifan Luo, Dingling Xu, Yukun Yan, Zhenghao Liu, Shi Yu, Ruobing Wang, Shuo Wang, Yishan Li, Nan Zhang, Xu Han, Zhiyuan Liu, Maosong Sun

FASPell: A Fast, Adaptable, Simple, Powerful Chinese Spell Checker Based On DAE-Decoder Paradigm (D19-55)

Challenge:	Existing evaluation metrics for RAG systems are lacking due to high costs of data construction and lack of factual accuracy.
Approach:	They propose a framework to evaluate RAG systems in specialized scenarios . they propose three new metrics to evaluate LLM-generated responses .
Outcome:	The proposed framework outperforms zero-shot and one-shot methods in terms of clarity, safety, conformity, and richness of generated samples.

Copied to clipboard

Yuzhong Hong, Xianguo Yu, Neng He, Nan Liu, Junhui Liu

AgentCourt: Simulating Court with Adversarial Evolvable Lawyer Agents (2025.findings-acl)

Challenge:	Existing spell checkers for Chinese are based on denoising autoencoder and decoder paradigms that require a small amount of data to be effective.
Approach:	They propose a Chinese spell checker based on a new paradigm which consists of a denoising autoencoder and a decoder.
Outcome:	The proposed spell checker is faster, more Adaptable to simplified and traditional Chinese texts and has a much simpler structure to be as much Powerful in error detection and correction.

Copied to clipboard

Guhong Chen, Liyang Fan, Zihan Gong, Nan Xie, Zixuan Li, Ziqiang Liu, Chengming Li, Qiang Qu, Hamid Alinejad-Rokny, Shiwen Ni, Min Yang

Robustness in NLP

Challenge:	Existing legal language models struggle with dynamic courtroom interactions, resulting in overfitting to standardized legal tasks.
Approach:	They propose a new adversarial evolutionary approach for agents that performs dynamic knowledge learning and evolution through structured adversarials in a simulated courtroom program.
Outcome:	The proposed approach outperforms existing LLM-based models in three critical dimensions: cognitive agility, professional knowledge, and logical rigor.

C-World: A Computer Use Agent Environment Creator (2026.acl-long)

Copied to clipboard

Ziqiao Xi, Shuang Liang, Qi Liu, Jiaqing Zhang, Letian Peng, Fang Nan, Meshal Nayim, Tianhui Zhang, Rishika Mundada, Lianhui Qin, Biwei Huang, Kun Zhou

Challenge:	C-World enables users to build agent environments on demand.
Approach:	They propose a system that enables users to build agent environments on demand.
Outcome:	The proposed system outperforms baselines on 119k samples and achieves Spearman = 0.883 ranking correlation with real execution.

ProQA: Structural Prompt-based Pre-training for Unified Question Answering (2022.naacl-main)

Copied to clipboard

Wanjun Zhong, Yifan Gao, Ning Ding, Yujia Qin, Zhiyuan Liu, Ming Zhou, Jiahai Wang, Jian Yin, Nan Duan

FIBER: Fill-in-the-Blanks as a Challenging Video Understanding Evaluation Framework (2022.acl-long)

Challenge:	Existing QA research on question answering is focused on specific question types, knowledge domains, or reasoning skills.
Approach:	They propose a unified QA paradigm that solves various tasks through a single model.
Outcome:	The proposed model improves QA-centric ability on 11 QA benchmarks.

Copied to clipboard

Santiago Castro, Ruoyao Wang, Pingxuan Huang, Ian Stewart, Oana Ignat, Nan Liu, Jonathan Stroud, Rada Mihalcea

GEM: A General Evaluation Benchmark for Multimodal Tasks (2021.findings-acl)

Challenge:	Existing video understanding evaluation frameworks that use fill-in-the-blanks do not reflect real-world tasks.
Approach:	They propose to use fill-in-the-blanks as a video understanding evaluation framework and introduce a novel dataset that collects multiple perspectives on the same video.
Outcome:	The proposed framework does not share the weaknesses of the current state-of-the-art language-informed video understanding tasks, namely: (1) video question answering using multiple-choice questions, where models perform relatively well because they exploit linguistic biases in the task formulation; (2) video captioning, which relies on an open-ended evaluation framework that is often inaccurate because system answers may be perceived as incorrect if they differ in form from the ground truth.

Copied to clipboard

Lin Su, Nan Duan, Edward Cui, Lei Ji, Chenfei Wu, Huaishao Luo, Yongfei Liu, Ming Zhong, Taroon Bharti, Arun Sacheti

Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Challenge:	Existing datasets that focus on natural language tasks are not considered as a general evaluation benchmark for multimodal tasks.
Approach:	They present a general evaluation benchmark for multimodal tasks, GEM 1 . they compare it with existing multimodal vision-language datasets .
Outcome:	The proposed model is compared with existing vision-language datasets focusing on natural language tasks . it is the largest vision-linguistic dataset covering image-language tasks and video-language task at the same time .

Copied to clipboard

Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, Tong Zhang

DART: Open-Domain Structured Data Record to Text Generation (2021.naacl-main)

Challenge:	Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach:	They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome:	The proposed method achieves the strongest alignment-forging Pareto front among competing methods.

Copied to clipboard

Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, Yangxiaokang Liu, Nadia Irwanto, Jessica Pan, Faiaz Rahman, Ahmad Zaidi, Mutethia Mutuma, Yasin Tarabar, Ankit Gupta, Tao Yu, Yi Chern Tan, Xi Victoria Lin, Caiming Xiong, Richard Socher, Nazneen Fatema Rajani

Structured Data in NLP

Challenge:	Data-to-text annotations can be costly when dealing with tables with nontrivial structures.
Approach:	They propose a procedure for extracting semantic triples from tables that encodes their structures by exploiting table headers and table title.
Outcome:	The proposed method exploits the semantic dependencies between table headers and title to extract semantic triples from tables.

ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)

Copied to clipboard

Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou

Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast (2023.acl-long)

Challenge:	Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism.
Approach:	They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning.
Outcome:	The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus.

Copied to clipboard

Yiduo Guo, Yaobo Liang, Dongyan Zhao, Bing Liu, Nan Duan

MetaScale: Test-Time Scaling with Evolving Meta-Thoughts (2026.findings-acl)

Challenge:	Existing research shows that a multilingual pre-trained language model fine-tuned with one (source) language performs well on downstream tasks for non-source languages . However, there is a clear performance gap between the source and non-sourced languages - this gap can be reduced by reducing forgetting.
Approach:	They propose a method to fine-tune a multilingual pre-trained language model fine- tuned with one (source) language and four training policies to address the performance gap.
Outcome:	The proposed method outperforms baselines on the XNLI dataset by a clear margin.

Copied to clipboard

Qin Liu, Wenxuan Zhou, Nan Xu, James Y. Huang, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

Leveraging Locality in Abstractive Text Summarization (2022.emnlp-main)

Challenge:	Existing approaches impose fixed cognitive structures that enhance performance in specific tasks but lack adaptability across diverse scenarios.
Approach:	They propose a test-time scaling framework based on meta-thoughts to improve performance . meta-thinkts are adaptive thinking strategies tailored to a given task .
Outcome:	Experimental results show that MetaScale outperforms standard inference approaches . it can scale more effectively with increasing sampling budgets and produces more structured responses .

Copied to clipboard

Yixin Liu, Ansong Ni, Linyong Nan, Budhaditya Deb, Chenguang Zhu, Ahmed Hassan Awadallah, Dragomir Radev

Summarization

Challenge:	Neural attention models have improved on many natural language processing tasks, but their quadratic memory complexity hinders their applications in long text summarization.
Approach:	They propose to use a restricted context to study locality in text summarization . they propose to employ a quadratic memory growth with respect to the input length .
Outcome:	The proposed model has better performance than baseline models with efficient attention modules.

Spotlighter: Revisiting Prompt Tuning from a Representative Mining View (2025.findings-emnlp)

Copied to clipboard

Yutong Gao, Maoyuan Shao, Xinyang Huang, Chuang Zhu, Yu Weng, Xuan Liu, Lijuan Sun, Guoshun Nan

Code Execution with Pre-trained Language Models (2023.findings-acl)

Challenge:	Spotlighter is a lightweight token-selection framework that enhances accuracy and efficiency in prompt tuning.
Approach:	They propose a token-selection framework that enhances accuracy and efficiency in prompt tuning by preserving only the top-scoring tokens for downstream prediction.
Outcome:	The proposed framework outperforms CLIP by up to 11.19% in harmonic mean accuracy and achieves 0.8K additional FPS, with only 21 extra parameters.

Copied to clipboard

Chenxiao Liu, Shuai Lu, Weizhu Chen, Daxin Jiang, Alexey Svyatkovskiy, Shengyu Fu, Neel Sundaresan, Nan Duan

Rethinking the Role of Prompting Strategies in LLM Test-Time Scaling: A Perspective of Probability Theory (2025.acl-long)

Challenge:	Pre-trained code intelligence models ignore the execution trace and only rely on source code and syntactic structures to understand code execution.
Approach:	They develop a mutation-based data augmentation technique to create a Python dataset and task for code execution that challenges existing models.
Outcome:	The proposed model outperforms existing models on code execution and shows its potential for zero-shot code-to-code search and text-to code generation.

Copied to clipboard

Yexiang Liu, Zekun Li, Zhi Fang, Nan Xu, Ran He, Tieniu Tan

Discontinuous Named Entity Recognition as Maximal Clique Discovery (2021.acl-long)

Challenge:	Recent studies have shown that scaling test-time compute can also effectively improve reasoning.
Approach:	They propose a probabilistic method to efficiently predict scaling performance and identify the best prompting strategy under large sampling times.
Outcome:	The proposed method significantly improves the scaling performance of majority voting on large language models.

Copied to clipboard

Yucheng Wang, Bowen Yu, Hongsong Zhu, Tingwen Liu, Nan Yu, Limin Sun

Information Extraction & Text Mining

Challenge:	Existing methods for named entity recognition break the recognition process into several sequential steps.
Approach:	They propose a method that breaks the recognition process into several sequential steps . they construct a segment graph for each sentence and a grid tagging scheme to learn it .
Outcome:	Experiments show that the proposed method outperforms the state-of-the-art model and achieves 5x speedup over the SOTA model.

S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (2025.acl-long)

Copied to clipboard

Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, Jia Li

Reasoning

Challenge:	Existing approaches to incentivize LLMs’ deep thinking abilities require large-scale data or significant training efforts.
Approach:	They introduce an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference.
Outcome:	The proposed framework outperforms models trained on long-CoT distilled data with 3.1k initialization samples and achieves an accuracy improvement of 51.0% to 81.6%.

Mask Attention Networks: Rethinking and Strengthen Transformer (2021.naacl-main)

Copied to clipboard

Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang

R2D2: Robust Data-to-Text with Replacement Detection (2022.emnlp-main)

Challenge:	Existing research explores to enhance the two sublayers separately to improve the capability of Transformer for text representation.
Approach:	They propose to combine SAN and Feed-Forward Networks to create a dynamic mask attention network with a learnable mask matrix which can model localness adaptively.
Outcome:	The proposed model outperforms the original Transformer on translation and text summarization tasks.

Copied to clipboard

Linyong Nan, Lorenzo Jaime Flores, Yilun Zhao, Yixin Liu, Luke Benson, Weijin Zou, Dragomir Radev

CodeTool: Enhancing Programmatic Tool Invocation of LLMs via Process Supervision (2025.acl-long)

Challenge:	Existing methods to mitigate unfaithful text generation are inadequate . data-to-text generation requires a structured input format .
Approach:	They propose a training framework that addresses unfaithful Data-to-Text generation by training a system as a generator and faithfulness discriminator with additional replacement detection and unlikelihood learning tasks.
Outcome:	The proposed training framework improves FeTaQA, LogicNLG, and ToTTo fidelity on D2T systems.

Copied to clipboard

YifeiLu YifeiLu, Fanghua Ye, Jian Li, Qiang Gao, Cheng Liu, Haibo Luo, Nan Du, Xiaolong Li, Feiliang Ren

Programming Languages in NLP

Challenge:	Existing approaches to tool invocation are often unnecessarily long and require lengthy reasoning paths.
Approach:	They propose a framework for stepwise code generation that improves LLM tool invocation . they incorporate two distinct process rewards: the On-the-spot and the Latent Reward .
Outcome:	The proposed framework improves LLM tool invocation by leveraging the concise nature of code.

Evaluating the Tradeoff Between Abstractiveness and Factuality in Abstractive Summarization (2023.findings-eacl)

Copied to clipboard

Markus Dreyer, Mengwen Liu, Feng Nan, Sandeep Atluri, Sujith Ravi

KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation (2022.findings-naacl)

Challenge:	Abstractive summarization models generate fluent and well-formed output but lack semantic faithfulness, or factuality, with respect to the input documents.
Approach:	They propose new factuality metrics that adjust for the degree of abstractiveness . they propose to visualize the rates of change in factual as we gradually increase abstractiveity .
Outcome:	The proposed models generate fluent and well-formed summaries but lack semantic faithfulness, or factuality, with respect to the input documents.

Copied to clipboard

Yongfei Liu, Chenfei Wu, Shao-Yen Tseng, Vasudev Lal, Xuming He, Nan Duan

Fair Abstractive Summarization of Diverse Perspectives (2024.naacl-long)

Challenge:	Existing vision-and-language pretraining approaches rely on external object detectors to encode images in a multi-modal transformer framework.
Approach:	They propose an object-aware end-to-end VLP framework which feeds image grid features from CNNs into the Transformer and learns the multi-modal representations jointly.
Outcome:	The proposed framework achieves competitive or superior performances on vision-language tasks.

Copied to clipboard

Yusen Zhang, Nan Zhang, Yixin Liu, Alexander Fabbri, Junru Liu, Ryo Kamoi, Xiaoxin Lu, Caiming Xiong, Jieyu Zhao, Dragomir Radev, Kathleen McKeown, Rui Zhang

Pruning as a Domain-specific LLM Extractor (2024.findings-naacl)

Challenge:	Existing work on summarization metrics and large language models has not explored fair abstractive summarizing.
Approach:	They propose four reference-free automatic metrics to measure the differences between target and source perspectives.
Outcome:	The proposed methods alleviate fair abstractive summarization on user-generated data.

Copied to clipboard

Nan Zhang, Yanchi Liu, Xujiang Zhao, Wei Cheng, Runxue Bao, Rui Zhang, Prasenjit Mitra, Haifeng Chen

Challenge:	Large Language Models (LLMs) have exhibited remarkable proficiency across a wide array of NLP tasks.
Approach:	They propose a method for pruning large language models using general or task-specific weights to extract a compressed, task-agnostic LLM.
Outcome:	The proposed method extracts a compressed, domain-specific, and task- agnostic LLM by identifying LLM weights that are pivotal for general capabilities, like linguistic capability and multi-task solving, and domain- specific knowledge.

Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior (2020.findings-emnlp)

Copied to clipboard

Zi Lin, Jeremiah Liu, Zi Yang, Nan Hua, Dan Roth

SimANS: Simple Ambiguous Negatives Sampling for Dense Text Retrieval (2022.emnlp-industry)

Challenge:	Spectral-normalized identity priors (SNIP) is a structured pruning approach for a Transformer model.
Approach:	They propose a structured pruning approach which penalizes an entire residual module toward an identity mapping.
Outcome:	The proposed method improves on 5 GLUE benchmark tasks while maintaining comparable performance.

Copied to clipboard

Kun Zhou, Yeyun Gong, Xiao Liu, Wayne Xin Zhao, Yelong Shen, Anlei Dong, Jingwen Lu, Rangan Majumder, Ji-rong Wen, Nan Duan

Information Retrieval

Challenge:	Existing methods for sapping negatives from large document pool suffer from the uninformative or false negative problem.
Approach:	They propose a method to sample negatives from a large document pool using a new sampling probability distribution.
Outcome:	The proposed method can be used to sample more ambiguous negatives on four public and one industry datasets.

Monotonic Paraphrasing Improves Generalization of Language Model Prompting (2024.findings-emnlp)

Copied to clipboard

Qin Liu, Fei Wang, Nan Xu, Tianyi Yan, Tao Meng, Muhao Chen

Is Graph Structure Necessary for Multi-hop Question Answering? (2020.emnlp-main)

Challenge:	Large language models (LLMs) have demonstrated remarkable proficiency in zero-shot decision making and instruction following.
Approach:	They propose an end-to-end decoding strategy that paraphrases given prompts or instructions into their lower perplexity counterparts based on an ensemble of a paraphrase LM for prompt rewriting, and a target LM that constrains the generation for lower perxity.
Outcome:	The proposed method can efficiently paraphrase the original prompt without altering its semantic meaning while decreasing the perplexity of each generation as calculated by the target LM.

Copied to clipboard

Nan Shao, Yiming Cui, Ting Liu, Shijin Wang, Guoping Hu

Structured Data in NLP

Challenge:	Existing studies focus on multi-hop question answering across multiple documents or paragraphs.
Approach:	They propose a graph neural network to deal with graph structure in textual multi-hop reasoning . they propose 'self-attention' and propose removing entire graph structure may not hurt the final results .
Outcome:	The proposed model shows that graph-attention or the entire graph structure can be replaced by self-attention . hotpotQA is a widely used benchmark for multi-hop question answering .

A2ATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantization (2025.findings-acl)

Copied to clipboard

Junhui He, Junna Xing, Nan Wang, Rui Xu, Shangyu Wu, Peng Zhou, Qiang Liu, Chun Jason Xue, Qingan Li

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Challenge:	Long context large language models (LLMs) pose significant challenges for efficient serving due to the large memory footprint and high access overhead of KV cache.
Approach:	They propose a retrieval-based method to reduce the memory footprint of LLMs . they propose Windowed Rotary Position Embedding and query-aware vector quantization .
Outcome:	The proposed method can achieve lower performance degradation with lower overhead compared to existing methods . it can reduce the memory footprint and access overhead of long context large language models .

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Nan Duan

Natural Language Interfaces

Challenge:	Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding.
Approach:	They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor .
Outcome:	The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks .

Streaming Hallucination Detection in Long Chain-of-Thought Reasoning (2026.findings-acl)

Copied to clipboard

Haolang Lu, Minghui Pan, Ripeng LI, Guoshun Nan, Jialin Zhuang, Zijie Zhao, Zhongxiang Sun, Kun Wang, Yang Liu

Reasoning

Challenge:	Long chain-of-thought reasoning improves performance of large language models, yet hallucinations in such settings often emerge subtly and propagate across reasoning steps.
Approach:	They propose to treat step-level hallucination judgments as local observations and introduce a cumulative prefix-level signal that tracks the global evolution of the reasoning state over the entire trajectory.
Outcome:	The proposed method enables streaming hallucination detection in long CoT reasoning, providing real-time, interpretable evidence.

CodeBERT: A Pre-Trained Model for Programming and Natural Languages (2020.findings-emnlp)

Copied to clipboard

Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, Ming Zhou

Semantic Parsing with Syntax- and Table-Aware SQL Generation (P18-1)

Challenge:	Large pre-trained models have improved performance on a variety of natural language processing tasks.
Approach:	They develop a bimodal pre-trained model for programming language (PL) and natural language (NL) it incorporates a hybrid objective function that detects replaced tokens from generators.
Outcome:	The proposed model performs better on two NL-PL applications by fine-tuning model parameters.

Copied to clipboard

Yibo Sun, Duyu Tang, Nan Duan, Jianshu Ji, Guihong Cao, Xiaocheng Feng, Bing Qin, Ting Liu, Ming Zhou

AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark (2025.acl-long)

Challenge:	Existing approaches generate a SQL query word-by-word but results are incorrect or not executable due to mismatch between question words and table contents.
Approach:	They propose a generative model to map natural language questions into SQL queries.
Outcome:	The proposed model significantly improves state-of-the-art execution accuracy from 69.0% to 74.4% on a large question- SQL dataset.

Copied to clipboard

Jianlyu Chen, Nan Wang, Chaofan Li, Bo Wang, Shitao Xiao, Han Xiao, Hao Liao, Defu Lian, Zheng Liu

Information Retrieval

Challenge:	Evaluation benchmarks based on predefined domains and human-labeled data face limitations in addressing evaluation needs for emerging domains.
Approach:	They propose an automated information retrieval benchmark based on predefined domains and human-labeled data . AIR-Bench is automated and Heterogeneous with three key features .
Outcome:	The proposed benchmarks are based on predefined domains and human-labeled data.

COFFEE: Counterfactual Fairness for Personalized Text Generation in Explainable Recommendation (2023.emnlp-main)

Copied to clipboard

Nan Wang, Qifan Wang, Yi-Chia Wang, Maziar Sanjabi, Jingzhou Liu, Hamed Firooz, Hongning Wang, Shaoliang Nie

Responsible & Trustworthy NLP

Challenge:	Personalized text generation (PTG) is a key component of our digital lives but can inadvertently associate different levels of linguistic quality with users’ protected attributes.
Approach:	They propose a framework to achieve measure-specific counterfactual fairness in explanation generation by focusing on one of the most studied settings: generating natural language explanations for recommendations.
Outcome:	The proposed framework achieves measure-specific counterfactual fairness in explanation generation.

MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering (2025.emnlp-main)

Copied to clipboard

Teng Lin, Yuyu Luo, Honglin Zhang, Jicheng Zhang, Chunlin Liu, Kaishun Wu, Nan Tang

Question Answering

Challenge:	Large Language Models (LLMs) and Retrieval-augmented Generation (RAG) systems show promise, but their performance on cross-document MEQA remains underexplored due to the lack of tailored benchmarks.
Approach:	They propose a scalable multi-document, multi-entity benchmark to evaluate LLMs' capacity to retrieve, consolidate, and reason over scattered and dense information.
Outcome:	The proposed benchmarks show that even advanced models achieve only 59% accuracy on MEBench.

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)

Copied to clipboard

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

Multilinguality

Challenge:	Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities.
Approach:	They propose a comprehensive benchmark covering 29 languages, built on an English benchmark.
Outcome:	The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark.

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)

Copied to clipboard

Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou

Multilinguality

Challenge:	XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios .
Approach:	They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora.
Outcome:	The proposed dataset is labeled in English and includes only natural language understanding tasks.

MedEureka: A Medical Domain Benchmark for Multi-Granularity and Multi-Data-Type Embedding-Based Retrieval (2025.findings-naacl)

Copied to clipboard

Yongqi Fan, Nan Wang, Kui Xue, Jingping Liu, Tong Ruan

SoftDedup: an Efficient Data Reweighting Method for Speeding Up Language Model Pre-training (2024.acl-long)

Challenge:	Embedding-based retrieval (EBR) is a mainstream approach in information retrieval.
Approach:	They propose an enriched benchmark to evaluate retrieval capabilities of embedding models . they use four levels of granularity and six types of medical texts to prompt instruction-fine-tuned embeddable models.
Outcome:	The proposed benchmark evaluates the retrieval capabilities of embedding models with multi-granularity and multi-data types.

Copied to clipboard

Nan He, Weichen Xiong, Hanwen Liu, Yi Liao, Lei Ding, Kai Zhang, Guohua Tang, Xiao Han, Yang Wei

LSDSCC: a Large Scale Domain-Specific Conversational Corpus for Response Generation with Diversity Oriented Evaluation Metrics (N18-1)

Challenge:	Current methods focus on detecting and removing duplicates, which risks the loss of valuable information and neglects the varying degrees of duplication.
Approach:	They propose a method that maintains dataset integrity while selectively reducing the sampling weight of data with high commonness.
Outcome:	The proposed method significantly improves training efficiency on deduplicated datasets and improves downstream accuracy by 1.77%.

Copied to clipboard

Zhen Xu, Nan Jiang, Bingquan Liu, Wenge Rong, Bowen Wu, Baoxun Wang, Zhuoran Wang, Xiaolong Wang

Dialogue Systems & Conversational Agents

Challenge:	Existing evaluation metrics for NRG models can't measure semantic relevance and diversity of generated results.
Approach:	They propose a large-scale domain-specific conversational corpus with preprocessing and cleansing procedures for model training and a testing set for measuring the diversity of generated results.
Outcome:	The proposed corpus can be taken as a new benchmark dataset for the NRG task.

DASA-Trans-STM: Adaptive Efficient Transformer for Short Text Matching using Data Augmentation and Semantic Awareness (2025.emnlp-main)

Copied to clipboard

Jiguo Liu, Chao Liu, Meimei Li, Nan Li, Shihao Gao, Dali Zhu

Responsible & Trustworthy NLP

Challenge:	Recent advances in large language models have shown impressive versatility across various tasks.
Approach:	They propose a novel adaptive Transformer for Chinese short text matching using data augmentation and semantic awareness.
Outcome:	The proposed model can deal with word ambiguity in Chinese on four available datasets.

Towards Robust Temporal Activity Localization Learning with Noisy Labels (2024.lrec-main)

Copied to clipboard

Daizong Liu, Xiaoye Qu, Xiang Fang, Jianfeng Dong, Pan Zhou, Guoshun Nan, Keke Tang, Wanlong Fang, Yu Cheng

Robustness in NLP

Challenge:	Existing methods for temporal activity localization are expensive and difficult to satisfy due to subjective labeling.
Approach:	They propose a new TAL setting where a TAL model should be robust to mixed training data with noisy moment boundaries.
Outcome:	The proposed method is significantly more robust to noisy training data than existing methods.

Rethinking Stateful Tool Use in Multi-Turn Dialogues: Benchmarks and Challenges (2025.findings-acl)

Copied to clipboard

Hongru Wang, Wenyu Huang, Yufei Wang, Yuanhao Xi, Jianqiao Lu, Huan Zhang, Nan Hu, Zeming Liu, Jeff Z. Pan, Kam-Fai Wong

Dialogue Systems & Conversational Agents

Challenge:	Existing benchmarks that assess Language Models (LMs) as Language Agents (LAs) for tool use focus on stateless, single-turn interactions or partial evaluations, overlooking the inherent stateful nature of interactions in multi-turn applications.
Approach:	They propose a multi-turn dialogue dataset with stateful tool interactions considering the whole life cycle of tool use across six key tasks in three stages . they also build VirtualMobile – an embodied virtual mobile evaluation environment to simulate API calls and assess the robustness of the created APIs.
Outcome:	The proposed dataset evaluates 13 open- and closed-source LLMs and provides detailed analysis at each stage.

NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation (2023.acl-long)

Copied to clipboard

Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Ming Gong, Lijuan Wang, Zicheng Liu, Houqiang Li, Nan Duan

Learning to Collaborate for Question Answering and Asking (N18-1)

Challenge:	Existing work generates long videos segment by segment sequentially, which is inefficient.
Approach:	They propose a Diffusion over Difference architecture for eXtremely Long video generation.
Outcome:	The proposed architecture reduces the average inference time from 7.55min to 26s (94.26%) and generates high-quality long videos with both global and local coherence.

Copied to clipboard

Duyu Tang, Nan Duan, Zhao Yan, Zhirui Zhang, Yibo Sun, Shujie Liu, Yuanhua Lv, Ming Zhou

Natural Language Interfaces

Challenge:	Question answering (QA) and question generation (QG) are closely related tasks.
Approach:	They propose a training algorithm that generalizes both Generative Adversarial Network and Generating Domain-Adaptive Nets under the question answering scenario.
Outcome:	The proposed training algorithm generalizes both Generative Adversarial Network (GAN) and Generating Domain-Adaptive Nets (GDAN) under the question answering scenario.

Document Modeling with Graph Attention Networks for Multi-grained Machine Reading Comprehension (2020.acl-main)

Copied to clipboard

Bo Zheng, Haoyang Wen, Yaobo Liang, Nan Duan, Wanxiang Che, Daxin Jiang, Ming Zhou, Ting Liu

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation (2020.emnlp-main)

Challenge:	Existing approaches to machine reading comprehension treat documents at their hierarchical nature, ignoring their dependencies.
Approach:	They propose a machine reading comprehension benchmark with two-grained answers . they use graph attention networks to model documents at their hierarchical nature .
Outcome:	The proposed framework outperforms existing systems at long and short answer criteria.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang, Jiancheng Lv, Nan Duan

Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation (2023.acl-long)

Challenge:	Existing methods for news headline generation focus on producing a single short sentence . et al., 2017; Gehrmann e.t., 2018; Zhong ee., 2019) focus on single-headline generation.
Approach:	They propose a method to generate multiple headlines with keyphrases of user interests . they propose generating multiple keyphrase-relevant headlines using a transformer decoder .
Outcome:	The proposed method achieves state-of-the-art in terms of quality and diversity.

Copied to clipboard

Yixin Liu, Alex Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, Dragomir Radev

Robustness in NLP

Challenge:	Existing studies for summarization evaluation exhibit low inter-annotator agreement or lack scale.
Approach:	They propose a modified summarization salience protocol based on fine-grained semantic units and a robust summarizing evaluation benchmark.
Outcome:	The proposed protocol is based on fine-grained semantic units and allows for high inter-annotator agreement.

QTSumm: Query-Focused Summarization over Tabular Data (2023.emnlp-main)

Copied to clipboard

Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Ruizhe Chen, Xiangru Tang, Yumo Xu, Dragomir Radev, Arman Cohan

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings (2026.acl-long)

Challenge:	Existing text generation systems that can provide accurate table summaries can facilitate more efficient access to relevant data insights.
Approach:	They propose a query-focused task where text generation models have to perform human-like reasoning and analysis over the given table to generate a tailored table summary.
Outcome:	The proposed method improves existing baselines on table-to-text generation and large language models by concatenating generated facts to the model input.

Copied to clipboard

Haonan Chen, Hong Liu, Yuping Luo, Liang Wang, Nan Yang, Furu Wei, Zhicheng Dou

Mixed-modality Representation Learning and Pre-training for Joint Table-and-Text Retrieval in OpenQA (2022.findings-emnlp)

Challenge:	Existing approaches to embed multimodal models face limitations such as suboptimal causal attention in VLMs and limited diversity in training objectives and data.
Approach:	They propose a framework for transforming pre-trained VLMs into bidirectional multimodal embedding models.
Outcome:	The proposed model improves performance across MMEB and ViDoRe-v2 benchmarks and exhibits strong scalability with both model size and training data on MMEF.

Copied to clipboard

Junjie Huang, Wanjun Zhong, Qian Liu, Ming Gong, Daxin Jiang, Nan Duan

Competition-Level Problems are Effective LLM Evaluators (2024.findings-acl)

Challenge:	Existing methods to retrieve evidences from corpus are difficult due to table-text discrepancy and data sparsity problem.
Approach:	They propose an optimized OpenQA Table-Text Retriever to retrieve tabular and textual evidences from tabular resources.
Outcome:	The proposed OpenQA Table-Text Retriever significantly outperforms existing methods on QA tasks.

Copied to clipboard

Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, Weizhu Chen

TED-TTS: Training-Free Intra-Utterance Emotion and Duration Control for Text-to-Speech Synthesis (2026.acl-long)

Challenge:	Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about their capabilities and the potential data contamination problem.
Approach:	They propose to evaluate the reasoning capabilities of large language models in solving recent competition-level programming problems in Codeforces.
Outcome:	The proposed model has experienced a cliff-like decline in problems after September 2021, which shows the potential data contamination and the challenges for any existing LLM to solve unseen complex reasoning problems.

Copied to clipboard

Qifan Liang, Yuansen Liu, Ruixin Wei, Nan Lu, Junchuan Zhao, Ye Wang

Allies: Prompting Large Language Model with Beam Search (2023.findings-emnlp)

Challenge:	Existing controllable Text-to-Speech methods limited to inter-utterance-level control . utterance expressiveness remains a challenge in building human-like TTS synthesis systems .
Approach:	They propose a training-free controllable framework for pretrained zero-shot TTS to enable intra-utterance emotion and duration expression.
Outcome:	The proposed framework achieves state-of-the-art intra-utterance consistency while maintaining baseline-level speech quality.

Copied to clipboard

Hao Sun, Xiao Liu, Yeyun Gong, Yan Zhang, Daxin Jiang, Linjun Yang, Nan Duan

Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents (2026.acl-long)

Challenge:	Existing methods to build LLMs with stacking are limited by their information coverage and low fault tolerance.
Approach:	They propose a method that leverages large language models to iteratively generate new queries from an input query.
Outcome:	The proposed method outperforms baselines on open-domain question answering benchmarks.

Copied to clipboard

Minzheng Wang, Run Luo, Yanbo Wang, Zichen Liu, Yuqiao Tan, Tao Tan, Nan Xu, Lu Wang, Wenji Mao

Concise Math Reasoning via Difficulty-Aware Distillation (2026.findings-acl)

Challenge:	Reinforcement Learning with Verifiable Rewards (RLVR) is effective for closed-ended tasks, but it is not applicable to open-ended social language games.
Approach:	They propose a method that uses a time-scaled evolutionary perception mechanism to detect impasse by quantifying dual-scale value baseline divergence alongside match entropy.
Outcome:	Experiments on multiple social language games show that the proposed method outperforms baselines and avoids policy degeneration.

Copied to clipboard

Yifan Wu, Jingze Shi, Bingheng Wu, Jiayi Zhang, Xiaotian Lin, Yizhang Zhu, Zhaoyang Yu, Bang Liu, Chenglin Wu, Nan Tang, Yuyu Luo

Numerical Reasoning

Challenge:	Human experts tackle difficult math problems by identifying and executing a few pivotal steps rather than listing every intermediate thought.
Approach:	They propose a method for producing training data that mirrors concise human reasoning by rewriting a problem's solution to retain only the essential steps.
Outcome:	The proposed method outperforms models trained on 800k long CoT and cuts training and inference costs.

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space (2020.emnlp-main)

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou