Papers by Nan Chen

MIKE: A New Benchmark for Fine-grained Multimodal Entity Knowledge Editing (2024.findings-acl)

Challenge:	Large Language Models (LLMs) have emerged as promising data science aids, assisting humans in data analysis and processing.
Approach:	They propose an evaluation paradigm and benchmarks that assess the performance of data science agents throughout the entire data science lifecycle.
Outcome:	The proposed evaluation paradigm streamlines dataset preparation, improves coverage, and expands benchmarking comprehensiveness.

Copied to clipboard

Jiaqi Li, Miaozeng Du, Chuanyi Zhang, Yongrui Chen, Nan Hu, Guilin Qi, Haiyun Jiang, Siyuan Cheng, Bozhong Tian

Question Generation from SQL Queries Improves Neural Semantic Parsing (D18-1)

Challenge:	Current benchmarks focus on coarse-grained knowledge, leaving the intricacies of fine-grounded knowledge unexplored.
Approach:	They propose a benchmark and dataset specifically designed for FG multimodal entity knowledge editing.
Outcome:	The proposed benchmark underscoring the complexity of FG knowledge editing in MLLMs.

Copied to clipboard

Daya Guo, Yibo Sun, Duyu Tang, Nan Duan, Jian Yin, Hong Chi, James Cao, Peng Chen, Ming Zhou

Title2Event: Benchmarking Open Event Extraction with a Large-scale Chinese Title Dataset (2022.emnlp-main)

Challenge:	Using question generation, we learn a semantic parser with 30% of the supervised training data.
Approach:	They propose to use question generation to learn a semantic parser with less supervised training data.
Outcome:	The proposed method improves the state-of-the-art model with less training data.

Copied to clipboard

Haolin Deng, Yanan Zhang, Yangfan Zhang, Wangyang Ying, Changlong Yu, Jun Gao, Wei Wang, Xiaoling Bai, Nan Yang, Jin Ma, Xiang Chen, Tianhua Zhou

Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking (2024.findings-naacl)

Challenge:	Existing EE datasets define fixed event types and design specific schemas for each of them, failing to cover diverse events emerging from the online text.
Approach:	They propose to use a sentence-level dataset to benchmark Open Event Extraction without restricting event types.
Outcome:	The proposed dataset contains more than 42,000 news titles in 34 topics collected from Chinese web pages.

Copied to clipboard

Nan Xu, Fei Wang, Ben Zhou, Bangzheng Li, Chaowei Xiao, Muhao Chen

Robustness in NLP

Challenge:	Large language models (LLMs) have demonstrated increasing power, but they also have vulnerabilities.
Approach:	They propose a black-box attack that targets the cognitive structure and processes of large language models (LLMs) they propose defending cognitive overload attacks from three perspectives.
Outcome:	The proposed attack is a black-box attack with no need for knowledge of model architecture or access to model weights.

Federated LoRA Fine-Tuning with Pipelined Error-Mitigated Aggregation and Matrix-Wise Freezing (2026.findings-acl)

Copied to clipboard

Haoran Wang, Xiong Wang, Yuqing Li, Jing Chen, Junyi Zhang, Nan Yan, Kun He, Wei Wang

Sentiment-Aware Word and Sentence Level Pre-training for Sentiment Analysis (2022.emnlp-main)

Challenge:	Existing methods for fine-tuning large language models often suffer from biased model aggregation and are hindered by significant communication and computation burden.
Approach:	They propose a Federated low-rank adaptation system for large language models that leverages pipelined error-mitigated model aggregation and adaptive matrix-wise parameter freezing to mitigate aggregations.
Outcome:	The proposed system improves time-to-target by 2.17-8.48 on real-world datasets.

Copied to clipboard

Shuai Fan, Chen Lin, Haonan Li, Zhenghao Lin, Jinsong Su, Hang Zhang, Yeyun Gong, JIan Guo, Nan Duan

Sentiment Analysis

Challenge:	Existing pre-trained language representation models (PLMs) capture sentiment information from word-level while under-considering sentence-level information.
Approach:	They propose a Sentiment-aware pre-trained language model with combined Word-level and Sentence-level Pre-training tasks that enhance the PLM’s knowledge about sentiment words.
Outcome:	The proposed model achieves state-of-the-art on various sentence-level and aspect-level sentiment classification benchmarks.

UnifEE: Unified Evidence Extraction for Fact Verification (2023.eacl-main)

Copied to clipboard

Nan Hu, Zirui Wu, Yuxuan Lai, Chen Zhang, Yansong Feng

OpenT2T: An Open-Source Toolkit for Table-to-Text Generation (2024.emnlp-demo)

Challenge:	Existing models extract evidence in both sentences and table cells from Wikipedia dumps, ignoring potential connections between them.
Approach:	They propose a model which uses a mixed evidence graph to extract the evidence in both formats without manually designed conversion rules.
Outcome:	The proposed model outperforms existing models and improves the verification step.

Copied to clipboard

Haowei Zhang, Shengyun Si, Yilun Zhao, Lujing Xie, Zhijian Xu, Lyuhao Chen, Linyong Nan, Pengcheng Wang, Xiangru Tang, Arman Cohan

Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding (2025.findings-acl)

Challenge:	Existing methods for table-to-text generation are limited and benchmarked on a limited number of datasets.
Approach:	They propose to use open-source tools to reproduce existing large language models for performance comparison and expedite the development of new models.
Outcome:	The proposed toolkit compares existing large language models on 9 table-to-text generation datasets and maintains a leaderboard to provide insights for future work.

Copied to clipboard

Zhanpeng Chen, Mingxiao Li, Ziyang Chen, Nan Du, Xiaolong Li, Yuexian Zou

From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning (2025.naacl-long)

Challenge:	Existing methods to encode visual positions inhibit the performance of vision-language Models (VLMs) however, language constitutes only one aspect of communication.
Approach:	They propose a method to assign visual position indexes from the periphery to the center and expand the central receptive field incrementally to enhance the perception of visual tokens within VLMs.
Outcome:	The proposed method reduces the relative distance between interrelated visual elements and instruction tokens, promoting a more rational allocation of attention weights and allowing for a multi-granularity perception of visual elements.

Copied to clipboard

Nan Xu, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

Low-Resource NLP

Challenge:	Motivated by in-context learning capabilities of Large Language Models (LLMs), multimodal LLMs with additional visual modality are also exhibited with similar ICL abilities when multiple image-text pairs are provided as demonstrations.
Approach:	They conduct systematic and principled evaluation of multimodal ICL for models of different scales on a broad spectrum of new yet critical tasks.
Outcome:	The proposed model performance improves on a broad spectrum of new yet critical tasks.

Contextual Fine-to-Coarse Distillation for Coarse-grained Response Selection in Open-Domain Conversations (2022.acl-long)

Copied to clipboard

Wei Chen, Yeyun Gong, Can Xu, Huang Hu, Bolun Yao, Zhongyu Wei, Zhihao Fan, Xiaowu Hu, Bartuer Zhou, Biao Cheng, Daxin Jiang, Nan Duan

ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)

Challenge:	Existing studies focus on coarse-grained response selection in retrieval-based dialogue systems.
Approach:	They propose a Contextual Fine-to-Coarse (CFC) distilled model for coarse-grained response selection in open-domain conversations.
Outcome:	The proposed model improves over baseline methods on two datasets based on the Reddit comments dump and Twitter corpus compared with baseline methods.

Copied to clipboard

Weizhen Qi, Yu Yan, Yeyun Gong, Dayiheng Liu, Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou

PPT: A Minor Language News Recommendation Model via Cross-Lingual Preference Pattern Transfer (2025.acl-long)

Challenge:	Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism.
Approach:	They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning.
Outcome:	The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus.

Copied to clipboard

Yiyang Zhang, Nan Chen

MetaScale: Test-Time Scaling with Evolving Meta-Thoughts (2026.findings-acl)

Challenge:	Existing systems for minor language news recommendation lack interaction with content . linguistic gap can lead to inaccurate modeling of minor language content despite strong English capability .
Approach:	They propose a minor language news recommendation model by cross-lingual preference pattern transfer . their model employs the widely used two-tower architecture and large language model as the backbone of the news encoder .
Outcome:	The proposed model outperforms existing models on 15 minor languages.

Copied to clipboard

Qin Liu, Wenxuan Zhou, Nan Xu, James Y. Huang, Fei Wang, Sheng Zhang, Hoifung Poon, Muhao Chen

LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection (2026.acl-long)

Challenge:	Existing approaches impose fixed cognitive structures that enhance performance in specific tasks but lack adaptability across diverse scenarios.
Approach:	They propose a test-time scaling framework based on meta-thoughts to improve performance . meta-thinkts are adaptive thinking strategies tailored to a given task .
Outcome:	Experimental results show that MetaScale outperforms standard inference approaches . it can scale more effectively with increasing sampling budgets and produces more structured responses .

Copied to clipboard

Cheng Xu, Changhong Jin, Yingjie Niu, Nan Yan, Yuke Mei, Shuhao Guan, Liming Chen, Tahar Kechadi

All You May Need for VQA are Image Captions (2022.naacl-main)

Challenge:	Current evaluation frameworks are static and vulnerable to benchmark data contamination . current models are ineffective at assessing reasoning under temporal uncertainty .
Approach:	They propose a live-based benchmark that simulates the real-world "fog of war" they propose evaluating models on their ability to reason with evolving, incomplete information .
Outcome:	The proposed model outperforms proprietary state-of-the-art models in classification and evidence mode . it also provides a component to monitor BDC explicitly .

Copied to clipboard

Soravit Changpinyo, Doron Kukliansy, Idan Szpektor, Xi Chen, Nan Ding, Radu Soricut

Code Execution with Pre-trained Language Models (2023.findings-acl)

Challenge:	Visual Question Answering (VQA) has benefited from increasingly sophisticated models, but has not enjoyed the same level of engagement in terms of data creation.
Approach:	They propose a method that automatically derives VQA examples at volume by leveraging existing image-caption annotations combined with neural models for textual question generation.
Outcome:	The proposed method improves state-of-the-art zero-shot accuracy by double digits and achieves robustness that lacks in the same model trained on human-annotated VQA data.

Copied to clipboard

Chenxiao Liu, Shuai Lu, Weizhu Chen, Daxin Jiang, Alexey Svyatkovskiy, Shengyu Fu, Neel Sundaresan, Nan Duan

Joint Generator-Ranker Learning for Natural Language Generation (2023.findings-acl)

Challenge:	Pre-trained code intelligence models ignore the execution trace and only rely on source code and syntactic structures to understand code execution.
Approach:	They develop a mutation-based data augmentation technique to create a Python dataset and task for code execution that challenges existing models.
Outcome:	The proposed model outperforms existing models on code execution and shows its potential for zero-shot code-to-code search and text-to code generation.

Copied to clipboard

Weizhou Shen, Yeyun Gong, Yelong Shen, Song Wang, Xiaojun Quan, Nan Duan, Weizhu Chen

S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (2025.acl-long)

Challenge:	Existing methods for text generation train the generator and ranker individually . existing methods neglect the mutual feedback that could enhance the quality of outputs .
Approach:	They propose a joint training algorithm that integrates the generator and ranker in a single framework.
Outcome:	The proposed algorithm surpasses existing methods on four public datasets across three common generation scenarios.

Copied to clipboard

Ruotian Ma, Peisong Wang, Cheng Liu, Xingyan Liu, Jiaqi Chen, Bang Zhang, Xin Zhou, Nan Du, Jia Li

On Diversified Preferences of Large Language Model Alignment (2024.findings-emnlp)

Challenge:	Existing approaches to incentivize LLMs’ deep thinking abilities require large-scale data or significant training efforts.
Approach:	They introduce an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference.
Outcome:	The proposed framework outperforms models trained on long-CoT distilled data with 3.1k initialization samples and achieves an accuracy improvement of 51.0% to 81.6%.

Copied to clipboard

Dun Zeng, Yong Dai, Pengyu Cheng, Longyue Wang, Tianhao Hu, Wanshun Chen, Nan Du, Zenglin Xu

DEMO: Reframing Dialogue Interaction with Fine-grained Element Modeling (2025.findings-acl)

Challenge:	Large language models (LLMs) can be fine tuned with human feedback, but human preferences can be diversified due to annotators’ different tastes, which hinders the effectiveness of LLM alignment methods.
Approach:	They propose a calibration error metric to evaluate large language models (LLMs) and a multi-objective reward learning method to enhance the calibration performance of RMs on shared preferences.
Outcome:	The proposed model can be adopted as a key calibration error and MORE can achieve superior alignment performance.

Copied to clipboard

Minzheng Wang, Xinghua Zhang, Kun Chen, Nan Xu, Haiyang Yu, Fei Huang, Wenji Mao, Yongbin Li

Harder Task Needs More Experts: Dynamic Routing in MoE Models (2024.acl-long)

Challenge:	Large language models (LLMs) enabled dialogue systems are one of the central modes in human-machine interaction.
Approach:	They propose a benchmark task for dialogue element MOdeling and Element Awareness and a new benchmark for dialogue agent interaction that allows the agent to model dialogue elements via imitation learning.
Outcome:	The proposed agent performs well in both dialogue element modeling and out-of-domain tasks.

Copied to clipboard

Quzhe Huang, Zhenwei An, Nan Zhuang, Mingxu Tao, Chen Zhang, Yang Jin, Kun Xu, Kun Xu, Liwei Chen, Songfang Huang, Yansong Feng

Pruning as a Domain-specific LLM Extractor (2024.findings-naacl)

Challenge:	Unlike existing MoE approaches that rely on fixed TopK Routing, our dynamic expert selection framework dynamically allocates experts based on the confidence level in expert selection for each input.
Approach:	They propose a dynamic expert selection framework that dynamically allocates experts based on the confidence level in expert selection for each input.
Outcome:	The proposed method achieves an average improvement of 0.7% with less than 90% activated parameters and outperforms dense models in QA and machine translation tasks.

Copied to clipboard

Nan Zhang, Yanchi Liu, Xujiang Zhao, Wei Cheng, Runxue Bao, Rui Zhang, Prasenjit Mitra, Haifeng Chen

Challenge:	Large Language Models (LLMs) have exhibited remarkable proficiency across a wide array of NLP tasks.
Approach:	They propose a method for pruning large language models using general or task-specific weights to extract a compressed, task-agnostic LLM.
Outcome:	The proposed method extracts a compressed, domain-specific, and task- agnostic LLM by identifying LLM weights that are pivotal for general capabilities, like linguistic capability and multi-task solving, and domain- specific knowledge.

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021.acl-demo)

Copied to clipboard

Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan

mDPO: Conditional Preference Optimization for Multimodal Large Language Models (2024.emnlp-main)

Challenge:	Existing models for pre-training are not convenient for users to find and set them up.
Approach:	They propose to extend ProphetNet into other domains and languages by pre-training models . they pre-train a cross-lingual generation model ProphetNet-Multi and a Chinese generation model .
Outcome:	The proposed models achieve new state-of-the-art on 10 benchmarks.

Copied to clipboard

Fei Wang, Wenxuan Zhou, James Y. Huang, Nan Xu, Sheng Zhang, Hoifung Poon, Muhao Chen

Monotonic Paraphrasing Improves Generalization of Language Model Prompting (2024.findings-emnlp)

Challenge:	Recent studies have attempted to apply DPO to multimodal scenarios but have found it challenging to achieve consistent improvement.
Approach:	They propose a multimodal DPO objective that prevents the over-prioritization of language-only preferences by also optimizing image preference.
Outcome:	The proposed method significantly improves performance on two multimodal LLMs of different sizes and three widely used benchmarks.

Copied to clipboard

Qin Liu, Fei Wang, Nan Xu, Tianyi Yan, Tao Meng, Muhao Chen

DUET: Joint Exploration of User–Item Profiles in Recommendation System (2026.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated remarkable proficiency in zero-shot decision making and instruction following.
Approach:	They propose an end-to-end decoding strategy that paraphrases given prompts or instructions into their lower perplexity counterparts based on an ensemble of a paraphrase LM for prompt rewriting, and a target LM that constrains the generation for lower perxity.
Outcome:	The proposed method can efficiently paraphrase the original prompt without altering its semantic meaning while decreasing the perplexity of each generation as calculated by the target LM.

Copied to clipboard

Yue Chen, Yifei Sun, Lu Wang, Fangkai Yang, Pu Zhao, Minjie Hong, Yifei Dong, Minghua He, Nan Hu, Jianjin Zhang, Zhiwei Dai, Yuefeng Zhan, Weihao Han, Hao Sun, Qingwei Lin, Weiwei Deng, Feng Sun, Qi Zhang, Saravan Rajmohan, Dongmei Zhang

Challenge:	Existing LLMs are opaque and difficult to interpret, resulting in limited interpretability.
Approach:	They propose an interaction-aware profile generator that jointly produces user and item profiles conditioned on both user history and item evidence.
Outcome:	The proposed model outperforms baselines on three real-world datasets.

RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations (2023.acl-long)

Copied to clipboard

Yilun Zhao, Chen Zhao, Linyong Nan, Zhenting Qi, Wenlin Zhang, Xiangru Tang, Boyu Mi, Dragomir Radev

FastSeq: Make Sequence Generation Faster (2021.acl-demo)

Challenge:	Existing Table QA models are vulnerable to task-specific perturbations, such as replacing key question entities or shuffling table columns.
Approach:	They propose to use large language models to generate adversarial examples to enhance training, which significantly improves the robustness of Table QA models.
Outcome:	The proposed model significantly improves on existing Table QA models against human-annotated adversarial perturbations.

Copied to clipboard

Yu Yan, Fei Hu, Jiusheng Chen, Nikhil Bhendawade, Ting Ye, Yeyun Gong, Nan Duan, Desheng Cui, Bingyu Chi, Ruofei Zhang

GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)

Challenge:	Transformer-based models have made tremendous impact in natural language generation, but inference speed is still a bottleneck due to large model size and intensive computing involved in auto-regressive decoding process.
Approach:	They propose an attention cache optimization, an efficient algorithm for detecting repeated n-grams, and an asynchronous generation pipeline with parallel I/O to accelerate sequence generation without loss of accuracy.
Outcome:	The proposed framework can accelerate the sequence generation by 4x to 9x with a simple one-line code change for a set of widely used and diverse models.

Copied to clipboard

Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan

More Than Spoken Words: Nonverbal Message Extraction and Generation (2023.emnlp-main)

Challenge:	Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models.
Approach:	They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks.
Outcome:	The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages.

Copied to clipboard

Dian Yu, Xiaoyang Wang, Wanshun Chen, Nan Du, Longyue Wang, Haitao Mi, Dong Yu

RikiNet: Reading Wikipedia Pages for Natural Question Answering (2020.acl-main)

Challenge:	Existing studies focus on extracting NMs from small-scale well-structured corpora such as movie scripts wherein NM is enclosed in parentheses by scriptwriters, which greatly decreases the difficulty of extraction.
Approach:	They propose to extract nonverbal messages (NMs) from written text and NMs from spoken text by using a semi-supervised learning algorithm.
Outcome:	The extracted NMs can generate more relevant, valid, and factually consistent NM than the purely supervised generator.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Nan Duan

KERAG: Knowledge-Enhanced Retrieval-Augmented Generation for Advanced Question Answering (2025.findings-emnlp)

Challenge:	Using Wikipedia pages to answer open-domain questions remains challenging in natural language understanding.
Approach:	They propose a model which reads Wikipedia pages for natural question answering . it uses a dynamic paragraph dual-attention reader and a cascaded answer predictor .
Outcome:	The proposed model outperforms the human model on the Natural Questions dataset . it achieves 74.3 F1 and 57.9 F1 on long-answer and short-answer tasks .

Copied to clipboard

Yushi Sun, Kai Sun, Yifan Ethan Xu, Xiao Yang, Xin Luna Dong, Nan Tang, Lei Chen

Structured Data in NLP

Challenge:	Traditional Knowledge Graph Question Answering (KGQA) methods rely on semantic parsing to retrieve knowledge strictly necessary for answer generation.
Approach:	They propose a retrieval-filtering-summarization pipeline that enhances QA coverage by retrieving a broader subgraph likely to contain relevant information.
Outcome:	The proposed pipeline surpasses state-of-the-art solutions by about 7% in quality and exceeds GPT-4o (Tool) by 10-21%.

Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy (2023.findings-emnlp)

Copied to clipboard

Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, Weizhu Chen

AIR-Bench: Automated Heterogeneous Information Retrieval Benchmark (2025.acl-long)

Challenge:	Recent work has proposed to improve relevance modeling by having large language models actively involved in retrieval, i.e., to guide retrieval with generation.
Approach:	They propose to have large language models actively involved in retrieval to guide retrieval with generation.
Outcome:	The proposed method synergizes retrieval and generation in an iterative manner, and can generate better results in subsequent iterations.

Copied to clipboard

Jianlyu Chen, Nan Wang, Chaofan Li, Bo Wang, Shitao Xiao, Han Xiao, Hao Liao, Defu Lian, Zheng Liu

Information Retrieval

Challenge:	Evaluation benchmarks based on predefined domains and human-labeled data face limitations in addressing evaluation needs for emerging domains.
Approach:	They propose an automated information retrieval benchmark based on predefined domains and human-labeled data . AIR-Bench is automated and Heterogeneous with three key features .
Outcome:	The proposed benchmarks are based on predefined domains and human-labeled data.

CoTKR: Chain-of-Thought Enhanced Knowledge Rewriting for Complex Knowledge Graph Question Answering (2024.emnlp-main)

Copied to clipboard

Yike Wu, Yi Huang, Nan Hu, Yuncheng Hua, Guilin Qi, Jiaoyan Chen, Jeff Pan

Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA (2024.emnlp-main)

Challenge:	Existing knowledge rewriting methods may include irrelevant information, omit crucial details, or fail to align with the question’s semantics.
Approach:	They propose a new rewriting method CoTKR for generating reasoning traces and corresponding knowledge in an interleaved manner, thereby mitigating the limitations of single-step knowledge rewrite.
Outcome:	The proposed method mitigates the limitations of single-step knowledge rewriting and bridges the preference gap between the knowledge reactor and the question answering (QA) model.

Copied to clipboard

Minzheng Wang, Longze Chen, Fu Cheng, Shengyi Liao, Xinghua Zhang, Bingli Wu, Haiyang Yu, Nan Xu, Lei Zhang, Run Luo, Yunshui Li, Min Yang, Fei Huang, Yongbin Li

Cooperative Denoising for Distantly Supervised Relation Extraction (C18-1)

Challenge:	Existing benchmarks for evaluating long-context language models employ irrelevant noise texts to artificially extend the length of test cases, diverging from the real-world scenarios of long-constituency applications.
Approach:	They propose a long-context benchmark, Loong, aligning with realistic scenarios through extended multi-document question answering (QA) .
Outcome:	The proposed model can scale up the context window of large language models to perform in-depth analysis of multiple long documents.

Copied to clipboard

Kai Lei, Daoyuan Chen, Yaliang Li, Nan Du, Min Yang, Wei Fan, Ying Shen

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)

Challenge:	Existing methods for distantly supervised relation extraction suffer from noisy labeling problem, which can severely degrade its performance.
Approach:	They propose a framework for distantly supervised relation extraction that leverages text corpus and knowledge graph and a cooperative module involving their mutual learning.
Outcome:	The proposed method reduces the noisy labels and achieves substantial improvement over the state-of-the-art methods.

Copied to clipboard

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

Multilinguality

Challenge:	Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities.
Approach:	They propose a comprehensive benchmark covering 29 languages, built on an English benchmark.
Outcome:	The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark.

XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation (2020.emnlp-main)

Copied to clipboard

Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, Ming Zhou

Multilinguality

Challenge:	XGLUE provides a benchmark dataset to train large-scale cross-lingual pre-trained models . XCLUE provides 11 diversified tasks that cover both understanding and generation scenarios .
Approach:	They introduce a new benchmark dataset to train large-scale cross-lingual pre-trained models using multilingual and bilingual corpora.
Outcome:	The proposed dataset is labeled in English and includes only natural language understanding tasks.

Dense Procedure Captioning in Narrated Instructional Videos (P19-1)

Copied to clipboard

Botian Shi, Lei Ji, Yaobo Liang, Nan Duan, Peng Chen, Zhendong Niu, Ming Zhou

Little Giants: Synthesizing High-Quality Embedding Data at Scale (2025.naacl-long)

Challenge:	Existing models for video dense captioning learn video segments and generate captions without considering transcripts.
Approach:	They propose a model to generate procedure captions from narrated instructional videos . they extract procedures by a cross-modality module and generate captions by encoding video frames and transcripts within each extracted procedure.
Outcome:	The proposed model can extract procedures from narrated instructional videos and generate procedure captions by encoding video frames and transcripts.

Copied to clipboard

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

MIND: Towards Immersive Psychological Healing with Multi-Agent Inner Dialogue (2025.findings-emnlp)

Challenge:	Synthetic data generation is an increasingly popular way of training models without the need for large, manually labeled datasets.
Approach:	They propose a framework that aligns open-source small models to efficiently generate large-scale embedding data.
Outcome:	The proposed framework outperforms state-of-the-art embedding models by using only 1/10 of the GPT API calls.

Copied to clipboard

Yujia Chen, Changsong Li, Yiming Wang, Tianjie Ju, Qingqing Xiao, Nan Zhang, Zifan Kong, Peng Wang, Binyu Yan

Extracting Symptoms and their Status from Clinical Conversations (P19-1)

Challenge:	Mental health issues are worsening in today’s competitive society, such as depression and anxiety.
Approach:	They propose a multi-agent inner dialogue paradigm that provides more immersive psychological healing environments.
Outcome:	The proposed paradigm provides more immersive psychological healing environments.

Copied to clipboard

Nan Du, Kai Chen, Anjuli Kannan, Linh Tran, Yuhui Chen, Izhak Shafran

Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing (2022.emnlp-main)

Challenge:	Existing models for extracting symptoms from clinical conversations are inherently difficult.
Approach:	They propose two new deep learning models tailored for a new application . they propose a hierarchical span-attribute tagging model and a sequence-to-sequence model .
Outcome:	The proposed models perform well under different conditions and are compared to existing models.

Copied to clipboard

Nan Xu, Fei Wang, Bangzheng Li, Mingtao Dong, Muhao Chen

Analytical Reasoning of Text (2022.findings-naacl)

Challenge:	Existing entity typing models are subject to spurious correlations due to shortcuts and biased training.
Approach:	They propose a method to augment existing model biases by combining spurious correlations with debiasedcounterparts to improve generalization.
Outcome:	The proposed method improves generalization of different entity typing models on the original and debiased test sets.

Copied to clipboard

Wanjun Zhong, Siyuan Wang, Duyu Tang, Zenan Xu, Daya Guo, Yining Chen, Jiahai Wang, Jian Yin, Ming Zhou, Nan Duan

Dense Retrieval as Indirect Supervision for Large-space Decision Making (2023.findings-emnlp)

Challenge:	Existing models with implicit reasoning ability struggle to solve analytical reasoning of text.
Approach:	They propose an approach to analyze text and use it to perform reasoning over it.
Outcome:	The proposed approach outperforms pre-trained models on an analysis of the Law School Admission Test dataset.

Copied to clipboard

Nan Xu, Fei Wang, Mingtao Dong, Muhao Chen

Information Retrieval

Challenge:	Dense Decision Retrieval (DDR) is a learning-to-retrieve task for discriminative natural language understanding (NLU) tasks with large label spaces.
Approach:	They propose a novel approach to learning large-space discriminative NLU tasks as a learning-to-retrieve task by adopting a dual-encoder architecture that learns to predict by retrieving from a decision thesaurus.
Outcome:	The proposed approach outperforms baselines greatly on multi-label classification tasks, 1.17% in F1 score ultra-fine entity typing, and 1.26% in accuracy on three few-shot intent classification tasks on average.

ToolExpNet: Optimizing Multi-Tool Selection in LLMs with Similarity and Dependency-Aware Experience Networks (2025.findings-acl)

Copied to clipboard

Zijing Zhang, Zhanpeng Chen, He Zhu, Ziyang Chen, Nan Du, Xiaolong Li

Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models (2024.findings-naacl)

Challenge:	Current empirical methods that focus on isolated tools learning struggle with accurate multi-tool selection due to issues like confusing similar tools and neglecting dependencies.
Approach:	They propose a tool-learning paradigm which integrates tools and trial-and-error experiences into a network characterized by semantic similarity and dependency relationships.
Outcome:	The proposed model outperforms existing methods on multiple real-world API datasets and significantly outperformed baselines.

Copied to clipboard

Jiashuo Sun, Yi Luo, Yeyun Gong, Chen Lin, Yelong Shen, Jian Guo, Nan Duan

ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering (2025.emnlp-main)

Challenge:	Chain-of-thought (CoT) prompting is a technique to enhance the reasoning abilities of Large language models (LLMs) however, the reasoning chains of demonstrations are observed to be prone to errors, which can lead to incorrect reasoning during inference.
Approach:	They propose an iterative bootstrapping technique to enhance the reasoning abilities of Large language models (LLMs) by generating a series of reasoning steps to obtain the answer, and using the reasoning chains as exemplars to demonstrate the task.
Outcome:	The proposed method improves the performance of Large language models (LLMs) on three reasoning tasks on ten datasets.

Copied to clipboard

Jingxuan Wei, Nan Xu, Junnan Zhu, null Haoyanni, Gaowei Wu, Qi Chen, Bihui Yu, Lei Wang

Hyperbolic Representations for Prompt Learning (2024.lrec-main)

Challenge:	Chart question answering (CQA) is a multimodal task for evaluating the reasoning capabilities of vision-language models.
Approach:	They propose a chart question answering benchmark that incorporates multilingual contexts and supports open-domain textual outputs.
Outcome:	The proposed framework outperforms the previous three common CQA paradigms: instruction-following, OCR-enhanced, and chain-of-thought.

Copied to clipboard

Nan Chen, Xiangdong Su, Feilong Bao

AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models (2024.findings-naacl)

Challenge:	Existing techniques to train only continuous prompts while freezing the language model have been developed.
Approach:	They propose to use hyperbolic space to model hierarchical relationships between prompts and inputs . they use a Poincaré disk to capture the hierarchic relationship between prompt and input .
Outcome:	The proposed approach reduces training time and storage for downstream tasks by reducing training costs.

Copied to clipboard

Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, Nan Duan

CodeRetriever: A Large Scale Contrastive Pre-Training Method for Code Search (2022.emnlp-main)

Challenge:	Traditional benchmarks for evaluating foundation models often fail to accurately represent their general abilities for human-centric tasks.
Approach:	They propose a bilingual benchmark to assess foundation models in the context of human-centric standardized exams such as college entrance exams, law school admission tests, and math competitions.
Outcome:	The proposed benchmark exceeds the average human performance on SAT, LSAT, and math competitions with 95% accuracy and 92.5% on the Chinese college entrance English exam.

Copied to clipboard

Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, Nan Duan

GPS: Genetic Prompt Search for Efficient Few-Shot Learning (2022.emnlp-main)

Challenge:	Existing code pre-training approaches often adopt (masked) language modeling as the training objective which targets on learning to predict (macked) tokens in a given code context.
Approach:	They propose a code-text contrastive learning model which learns function-level code semantic representations through large-scale code corpus.
Outcome:	The proposed model achieves new state-of-the-art with significant improvement over existing pre-trained models on eleven domain/language-specific code search tasks with six programming languages in different code granularity.

Copied to clipboard

Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Wang Yanggang, Haiyu Li, Zhilin Yang

Beyond Query Bias: Candidate-Aware Iterative Refinement for Zero-Shot Composed Image Retrieval (2026.findings-acl)

Challenge:	Pretrained language models are often finetuned for downstream tasks, which has been shown to improve performance over non-pretrained models.
Approach:	They propose a genetic algorithm to automatically search for the best prompt for few-shot learning with pretrained language models by gradient-free algorithm.
Outcome:	Experiments on diverse datasets show that the proposed method outperforms manual prompts by 2.6 points.

Copied to clipboard

Nan Sun, Jing Tang, Lei Sun, Rui Chen, Yuxing Lu, Xiangxiang Chu, Hefei Ling, Yujun Cai

Soft-Labeled Contrastive Pre-Training for Function-Level Code Representation (2022.findings-emnlp)

Challenge:	Existing methods to retrieve target images suffer from inherent cognitive bias due to unknown candidate distribution.
Approach:	They propose a training-free framework that reframes ZS-CIR as a self-correcting process . they propose to use retrieved results as feedback to perceive the candidate distribution .
Outcome:	Experiments on public benchmarks show that CoRR outperforms other SOTA methods.

Copied to clipboard

Xiaonan Li, Daya Guo, Yeyun Gong, Yun Lin, Yelong Shen, Xipeng Qiu, Daxin Jiang, Weizhu Chen, Nan Duan

mmE5: Improving Multimodal Multilingual Embeddings via High-quality Synthetic Data (2025.findings-acl)

Challenge:	Existing methods for contrastive pre-training ignore the relevance between codes in large code corpus.
Approach:	They propose a Soft-labeled contrastive pre-training framework with positive sample construction methods to learn functional-level code representation.
Outcome:	The proposed framework can obtain fine-grained soft-labels through an iterative adversarial manner and use them to learn better code representation.

Copied to clipboard

Haonan Chen, Liang Wang, Nan Yang, Yutao Zhu, Ziliang Zhao, Furu Wei, Zhicheng Dou

The Medical Scribe: Corpus Development and Model Performance Analyses (2020.lrec-1)

Challenge:	Multimodal embedding models encode multimedia inputs into latent vector representations.
Approach:	They propose to synthesize multimodal multilingual data using a multimodal large language model . they identify three criteria for high-quality synthetic multimodal data .
Outcome:	The proposed model outperforms existing models on the MMEB Benchmark and the XTD benchmark.

Copied to clipboard

Izhak Shafran, Nan Du, Linh Tran, Amanda Perry, Lauren Keyes, Mark Knichel, Ashley Domin, Lei Huang, Yu-hui Chen, Gang Li, Mingqiu Wang, Laurent El Shafey, Hagen Soltau, Justin Stuart Paul

Challenge:	Existing tools to assist in clinical note generation using audio of provider-patient encounters are lacking.
Approach:	They develop an annotation scheme to extract relevant clinical concepts from audio of provider-patient encounters and train a state-of-the-art tagging model.
Outcome:	The proposed model is more useful than the F-scores reflect and can be used in clinical notes.

Can LLMs Evaluate Complex Attribution in QA? Automatic Benchmarking using Knowledge Graphs (2025.acl-long)

Copied to clipboard

Nan Hu, Jiaoyan Chen, Yike Wu, Guilin Qi, Hongru Wang, Sheng Bi, Yongrui Chen, Tongtong Wu, Jeff Z. Pan

CEO: Corpus-based Open-Domain Event Ontology Induction (2024.findings-eacl)

Challenge:	Attributed Question Answering (AQA) has attracted wide attention, but there are several limitations in evaluating the attributions.
Approach:	They propose a large-scale benchmark containing comprehensive attribution categories . they compare 25 automatic evaluators with human evaluers and tested LLM evalators .
Outcome:	The proposed method can compare attributions with subtle differences and provide feedback to improve them.

Copied to clipboard

Nan Xu, Hongming Zhang, Jianshu Chen

Knowledge Representation

Challenge:	Existing event-centric NLP models restrict their generalization capabilities by limiting the pre-defined ontology.
Approach:	They propose a Corpus-based Event Ontology induction model to relax the restriction imposed by pre-defined ontologies.
Outcome:	The proposed model can induce a hierarchical event ontology with meaningful names on eleven open-domain corpora, making it more trustworthy and easier to be further curated.

QTSumm: Query-Focused Summarization over Tabular Data (2023.emnlp-main)

Copied to clipboard

Yilun Zhao, Zhenting Qi, Linyong Nan, Boyu Mi, Yixin Liu, Weijin Zou, Simeng Han, Ruizhe Chen, Xiangru Tang, Yumo Xu, Dragomir Radev, Arman Cohan

MINTQA: A Multi-Hop Question Answering Benchmark for Evaluating LLMs on New and Long-tail Knowledge (2026.acl-long)

Challenge:	Existing text generation systems that can provide accurate table summaries can facilitate more efficient access to relevant data insights.
Approach:	They propose a query-focused task where text generation models have to perform human-like reasoning and analysis over the given table to generate a tailored table summary.
Outcome:	The proposed method improves existing baselines on table-to-text generation and large language models by concatenating generated facts to the model input.

Copied to clipboard

Jie He, Nan Hu, Wanqiu Long, Jiaoyan Chen, Jeff Z. Pan

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings (2026.acl-long)

Challenge:	Existing studies have shown that large language models can handle knowledge with varying familiarity.
Approach:	They propose a benchmark to evaluate multi-hop question answering on new and tail knowledge . they use RAG to integrate external knowledge into large language models .
Outcome:	The proposed benchmark evaluates the multi-hop reasoning ability of large language models . it primarily evaluates their ability to handle knowledge with different levels of familiarity .

Copied to clipboard

Haonan Chen, Hong Liu, Yuping Luo, Liang Wang, Nan Yang, Furu Wei, Zhicheng Dou

Competition-Level Problems are Effective LLM Evaluators (2024.findings-acl)

Challenge:	Existing approaches to embed multimodal models face limitations such as suboptimal causal attention in VLMs and limited diversity in training objectives and data.
Approach:	They propose a framework for transforming pre-trained VLMs into bidirectional multimodal embedding models.
Outcome:	The proposed model improves performance across MMEB and ViDoRe-v2 benchmarks and exhibits strong scalability with both model size and training data on MMEF.

Copied to clipboard

Yiming Huang, Zhenghao Lin, Xiao Liu, Yeyun Gong, Shuai Lu, Fangyu Lei, Yaobo Liang, Yelong Shen, Chen Lin, Nan Duan, Weizhu Chen

Graph Neural News Recommendation with Unsupervised Preference Disentanglement (2020.acl-main)

Challenge:	Large language models (LLMs) have demonstrated impressive reasoning capabilities, yet there is ongoing debate about their capabilities and the potential data contamination problem.
Approach:	They propose to evaluate the reasoning capabilities of large language models in solving recent competition-level programming problems in Codeforces.
Outcome:	The proposed model has experienced a cliff-like decline in problems after September 2021, which shows the potential data contamination and the challenges for any existing LLM to solve unseen complex reasoning problems.

Copied to clipboard

Linmei Hu, Siyong Xu, Chen Li, Cheng Yang, Chuan Shi, Nan Duan, Xing Xie, Ming Zhou

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation (2022.acl-long)

Challenge:	Existing methods to learn informative user and news representations fail to consider high-order connectivity underlying the user-news interactions.
Approach:	They propose a novel Graph Neural News Recommendation model with Unsupervised Preference Disentanglement which can encode high-order relationships into user and news representations by information propagation along the graph.
Outcome:	The proposed model can encode high-order relationships into user and news representations by information propagation along the graph and disentangle latent preference factors by a neighborhood routing algorithm.

Copied to clipboard

Wei Chen, Yeyun Gong, Song Wang, Bolun Yao, Weizhen Qi, Zhongyu Wei, Xiaowu Hu, Bartuer Zhou, Yi Mao, Weizhu Chen, Biao Cheng, Nan Duan

ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization (2022.findings-emnlp)

Challenge:	Existing pre-trained dialog models shed light on various downstream tasks in natural language processing (NLP).
Approach:	They propose a dialog pre-training framework that introduces latent variables into the enhanced encoder-decoder pre-train framework to increase relevance and diversity of responses.
Outcome:	The proposed model achieves state-of-the-art on personaChat, DailyDialog, and DSTC7-AVSD datasets.

Copied to clipboard

Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Wang Yanggang, Haiyu Li, Zhilin Yang

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space (2020.emnlp-main)

Challenge:	a recent study shows that task scaling can be an efficient alternative to model scaling.
Approach:	They propose a multitask pretraining approach ZeroPrompt for zero-shot generalization . they focus on task scaling and zero-shooting to improve model performance .
Outcome:	The proposed approach improves zero-shot generalization efficiency by 30 times with task scaling.

Copied to clipboard

Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan, Ming Zhou