Papers by Dong Li

We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning? (2025.acl-long)

Challenge:	Existing models cannot fully recognize the specific expressions given by users due to the informality and diversity of natural language expressions.
Approach:	They propose a Heterogeneous User History graph convolution network which utilizes the user’s historical answers grouped by DA labels as additional clues to recognize the DA label of utterances.
Outcome:	The proposed model outperforms the state-of-the-art methods on two benchmark datasets and shows that it integrates user’s historical answers.

Copied to clipboard

Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Chong Sun, Xiaoshuai Song, Jiapeng Wang, Zhuoma GongQue, Shanglin Lei, YiFan Zhang, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Xiao Zong, Yida Xu, Peiqing Yang, Zhimin Bao, Muxi Diao, Chen Li, Honggang Zhang

From Speculation Detection to Trustworthy Relational Tuples in Information Extraction (2023.findings-emnlp)

Challenge:	Existing benchmarks focus more on end-to-end performance, but neglect the underlying principles of knowledge acquisition and generalization.
Approach:	They propose a benchmark specifically designed to explore the problem-solving principles by decomposing 6.5K visual math problems into 10.9K step-level questions for evaluation.
Outcome:	The proposed benchmark covers 6.5K visual math problems and 10.9K step-level questions spanning 5 layers of knowledge granularity and 67 hierarchical knowledge concepts.

Copied to clipboard

Kuicai Dong, Aixin Sun, Jung-jae Kim, Xiaoli Li

CODERL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment (2026.acl-long)

Challenge:	Existing studies on speculation detection are defined at sentence level, but not all factual tuples extracted from a sentence are speculative.
Approach:	They propose to study speculations in OIE tuples and determine whether a tample is speculative.
Outcome:	The proposed model is based on the LSOIE dataset and provides labels for speculative tuples.

Copied to clipboard

Xue Jiang, Yihong Dong, Mengyang Liu, Deng Hongyi, Tian Wang, Yongding Tao, Zhi Jin, Wenpin Jiao, Ge Li

When to Trust Tools? Adaptive Tool Trust Calibration For Tool-Integrated Math Reasoning (2026.findings-acl)

Challenge:	Large Language Models excel at code generation by learning from vast code corpora, but a fundamental semantic gap remains between training on textual patterns and the goal of functional correctness . reinforcement learning with verifiable rewards (RLVR) approaches are inefficient for establishing a well-aligned connection between the textual representation of code and its execution semantics.
Approach:	They propose a novel approach that integrates execution semantics alignment into the RLVR training pipeline for code generation.
Outcome:	The proposed model outperforms baseline training and RLVR and shows strong applicability across RL and LLMs.

Copied to clipboard

Ruotao Xu, Yixin Ji, Yu Luo, Jinpeng Li, Dong Li, Peifeng Li, Juntao Li, Min Zhang

Harvesting and Refining Question-Answer Pairs for Unsupervised QA (2020.acl-main)

Challenge:	Large reasoning models have performance enhancements but still suffer from shortcomings due to limitations of the underlying language models.
Approach:	They propose a framework that allows the model to choose when to trust or ignore the tool results based on the confidence score of generated code blocks.
Outcome:	The proposed framework reduces the "Tool Ignored" issue by 4.1% to 7.5%.

Copied to clipboard

Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

Uncertainty Quantification for Clinical Outcome Predictions with (Large) Language Models (2025.findings-naacl)

Challenge:	Recent research attempts to extend unsupervised question answering to settings with few or no labeled data available.
Approach:	They propose two approaches to improve unsupervised question answering . first, they harvest lexically and syntactically divergent Wikipedia questions to automatically construct a corpus of question-answer pairs . second, they take advantage of the QA model to extract more appropriate answers .
Outcome:	The proposed approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models.

Copied to clipboard

Zizhang Chen, Peizhao Li, Xiaomeng Dong, Pengyu Hong

Negative Matters: Multi-Granularity Hard-Negative Synthesis and Anchor-Token-Aware Pooling for Enhanced Text Embeddings (2025.acl-long)

Challenge:	Language models (LMs) have significant potential for clinical prediction tasks . however, unreliable decisions can result in significant costs due to compromised patient safety and ethical concerns .
Approach:	They propose to combine ensembling and multi-tasking approaches to reduce uncertainty in EHRs by using multi-tapping methods.
Outcome:	The proposed framework reduces model uncertainty in white-box and black-box settings, and improves model transparency in both settings.

Copied to clipboard

Tengyu Pan, Zhichao Duan, Zhenyu Li, Bowen Dong, Ning Liu, Xiuxing Li, Jianyong Wang

Representation Learning

Challenge:	Text embedding models are used for various natural language processing tasks such as sentiment analysis, text clustering, and content-based information retrieval.
Approach:	They propose a synthesis framework that leverages large language models to generate diverse negative samples with varying levels of similarity with the query.
Outcome:	The proposed framework achieves state-of-the-art performance surpassing existing synthesis strategies with synthetic data and when combined with public retrieval datasets.

Pay Attention to Implicit Attribute Values: A Multi-modal Generative Framework for AVE Task (2023.findings-acl)

Copied to clipboard

Yupeng Zhang, Shensi Wang, Peiguang Li, Guanting Dong, Sirui Wang, Yunsen Xian, Zhoujun Li, Hongzhi Zhang

NeurST: Neural Speech Translation Toolkit (2021.acl-demo)

Challenge:	Existing approaches to extract attribute values from product descriptions are incomplete and noisy due to the tedious nature of this task.
Approach:	They propose a framework to extract attributes from product descriptions to acquire implicit attributes in addition to the explicit ones.
Outcome:	The proposed framework outperforms existing methods on the extraction of implicit attribute values while achieving comparable performance for the explicit ones.

Copied to clipboard

Chengqi Zhao, Mingxuan Wang, Qianqian Dong, Rong Ye, Lei Li

Language Models as Inductive Reasoners (2024.eacl-long)

Challenge:	a toolkit for speech translation is available for free and provides step-by-step recipes for feature extraction, data preprocessing, distributed training, and evaluation.
Approach:	They propose to use NeurST to facilitate speech translation research for NLP researchers . they show experimental results for different benchmark datasets which can be regarded as reliable baselines .
Outcome:	The proposed framework provides reliable benchmarks for speech translation research.

Copied to clipboard

Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, Furu Wei

Multi-Document Scientific Summarization from a Knowledge Graph-Centric View (2022.coling-1)

Challenge:	Inductive reasoning is a core component of human intelligence.
Approach:	They propose a task to induce natural language rules from natural language facts using natural language as representation for knowledge instead of formal language.
Outcome:	The proposed task surpasses baselines in both automatic and human evaluations.

Copied to clipboard

Pancheng Wang, Shasha Li, Kunyuan Pang, Liangliang He, Dong Li, Jintao Tang, Ting Wang

Towards Storage-Efficient Visual Document Retrieval: An Empirical Study on Reducing Patch-Level Embeddings (2025.findings-acl)

Challenge:	Multi-Document Scientific Summarization (MDSS) aims to produce concise and concise summaries for clusters of topic-relevant scientific papers.
Approach:	They propose a model that incorporates knowledge graphs into paper encoding and decoding processes and propose 'decoder' for generating knowledge graph information of summary in the form of descriptive sentences.
Outcome:	The proposed architecture improves on baselines on the Multi-Xscience dataset.

Copied to clipboard

Yubo Ma, Jinsong Li, Yuhang Zang, Xiaobao Wu, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Jiaqi Wang, Yixin Cao, Aixin Sun

Representation Learning

Challenge:	Visualized Document Retrieval (VDR) uses large vision-language models to encode document pages into embeddings.
Approach:	They evaluate methods to reduce patch embeddings per page while minimizing performance degradation.
Outcome:	The proposed method maintains 98.2% of retrieval performance with only 11.8% of original memory usage and preserves 94.6% effectiveness at 2% memory footprint.

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model (2025.findings-naacl)

Copied to clipboard

Zhaowei Li, Wei Wang, YiQing Cai, Qi Xu, Pengyu Wang, Dong Zhang, Hang Song, Botian Jiang, Zhida Huang, Tao Wang

IntentCoding: Amplifying User Intent in Code Generation (2026.findings-acl)

Challenge:	Representative models like LLaVA and MiniGPT-4 have great capabilities in various tasks.
Approach:	They propose a unified model to represent various multi-modal tasks using a single representation.
Outcome:	The proposed model outperforms existing models in a variety of tasks while maintaining generality and scalability.

Copied to clipboard

Zheng Fang, Yihong Dong, Lili Mou, Dongming Jin, Zhi Jin, Ge Li

Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization (2026.acl-long)

Challenge:	IntentCoding captures the influence of user intent by masking out the intent, and integrates seamlessly with existing decoding procedures.
Approach:	They propose a decoding strategy that captures the influence of user intent by masking out the intent and applies a multi-strength ensemble mechanism to amplify the effect of user intention during generation.
Outcome:	The proposed model significantly improves both constraint satisfaction and functional correctness compared to greedy decoding approaches.

Copied to clipboard

Tian Xueyun, MingHua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang, Huawei Shen

Joint Representation Learning of Cross-lingual Words and Entities via Attentive Distant Supervision (D18-1)

Challenge:	Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models.
Approach:	They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains .
Outcome:	The proposed scheme yields 5.51% OOD gain over positive-only training.

Copied to clipboard

Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Chengjiang Li, Xu Chen, Tiansi Dong

Cross-domain NER with Generated Task-Oriented Knowledge: An Empirical Study from Information Density Perspective (2024.emnlp-main)

Challenge:	Existing methods for learning word and entity representations in monolingual settings are limited.
Approach:	They propose a method for joint representation learning of cross-lingual words and entities that captures mutually complementary knowledge and enables cross-linguistic inferences.
Outcome:	The proposed method captures mutually complementary knowledge and enables cross-lingual inferences among knowledge bases and texts.

Copied to clipboard

Zhihao Zhang, Sophia Lee, Junshuang Wu, Dong Zhang, Shoushan Li, Erik Cambria, Guodong Zhou

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-Context QA (2025.findings-acl)

Challenge:	Cross-domain Named Entity Recognition (CDNER) is crucial for Knowledge Graph (KG) construction and natural language processing (NLP)
Approach:	They propose to automatically generate task-oriented knowledge using large language models (LLMs) and then employ task-orientated pre-training (TOPT) to facilitate domain adaptation.
Outcome:	The proposed model can learn to distinguish between different entities and improve its domain adaptation.

Copied to clipboard

Jiajie Zhang, Yushi Bai, Xin Lv, Wanjun Gu, Danqing Liu, Minhao Zou, Shulin Cao, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

Question Answering

Challenge:	Current long-context large language models lack citations to support their responses, making verification difficult due to potential hallucinations.
Approach:	They propose to use off-the-shelf LLMs to automatically construct long-context QA instances with precise sentence-level citations and leverage this pipeline to construct a large-scale SFT dataset for LQAC.
Outcome:	The proposed pipeline can generate responses with fine-grained citations on the fly, surpassing existing models including GPT-4o.

FB-Bench: A Fine-Grained Multi-Task Benchmark for Evaluating LLMs’ Responsiveness to Human Feedback (2025.emnlp-main)

Copied to clipboard

Youquan Li, Miao Zheng, Fan Yang, Guosheng Dong, Bin Cui, Weipeng Chen, Zenan Zhou, Wentao Zhang

Tab-Cleaner: Weakly Supervised Tabular Data Cleaning via Pre-training for E-commerce Catalog (2023.acl-industry)

Challenge:	Existing research focuses on benchmarking LLMs in single-turn dialogues, neglecting the nuanced nature of human feedback within real-world usage scenarios.
Approach:	They propose a fine-grained, multi-task benchmark designed to evaluate LLMs’ responsiveness to human feedback under real-world usage scenarios in Chinese.
Outcome:	The proposed benchmarks show that human feedback can significantly impact LLMs’ responsiveness in real-world usage scenarios.

Copied to clipboard

Kewei Cheng, Xian Li, Zhengyang Wang, Chenwei Zhang, Binxuan Huang, Yifan Ethan Xu, Xin Luna Dong, Yizhou Sun

SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities (2022.acl-long)

Challenge:	Existing methods for analyzing textual attributes in product catalogs are not effective on structured tabular data since they are trained on free-form natural language texts.
Approach:	They propose a model to handle error detection over tabular data following a pre-training paradigm.
Outcome:	The proposed model improves on a real-world Amazon Product Catalog table by 16% over state-of-the-art methods and by 11% on PR AUC over attribute value validation task.

Copied to clipboard

Hsiang-Sheng Tsai, Heng-Jui Chang, Wen-Chin Huang, Zili Huang, Kushal Lakhotia, Shu-wen Yang, Shuyan Dong, Andy Liu, Cheng-I Lai, Jiatong Shi, Xuankai Chang, Phil Hall, Hsuan-Jui Chen, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee

Speech & Audio in NLP

Challenge:	Existing evaluation methods for transfer learning are limited in speech research . authors show that pre-trained models transfer well across multiple tasks .
Approach:	They propose a benchmark to evaluate pre-trained models by increasing task diversity and difficulty over SUPERB.
Outcome:	The proposed benchmark increases task diversity and difficulty over SUPERB-SG.

CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation (2022.findings-emnlp)

Copied to clipboard

Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei

P-INT: A Path-based Interaction Model for Few-shot Knowledge Graph Completion (2021.findings-emnlp)

Challenge:	Named entity recognition (NER) suffers from the scarcity of annotated training data, especially for low-resource languages without labeled data.
Approach:	They propose a cross-lingual entity projection framework to enable zero-shot cross-linguistic NER with the help of a multilingual labeled sequence translation model.
Outcome:	The proposed method outperforms the baseline method on two benchmarks by a large margin of +3 7 F1 scores and achieves state-of-the-art performance.

Copied to clipboard

Jingwen Xu, Jing Zhang, Xirui Ke, Yuxiao Dong, Hong Chen, Cuiping Li, Yongbin Liu

CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Control (2025.findings-acl)

Challenge:	Existing methods to encode and match entity pairs have only a few observed reference entity pairs.
Approach:	They propose a model that infers and leverages paths that can expressively encode the relation of two entities.
Outcome:	The proposed model outperforms the state-of-the-art models by 11.2– 14.2% in terms of Hits@1.

Copied to clipboard

Liu Huanshuo, Hao Zhang, Zhijiang Guo, Jing Wang, Kuicai Dong, Xiangyang Li, Yi Quan Lee, Cong Zhang, Yong Liu

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA (2022.acl-long)

Challenge:	Existing methods focus on detecting LLM’s confidence via statistical uncertainty.
Approach:	They propose to use a representation perspective to solve adaptive RAG by enabling dynamic retrieval during generation and enabling retrieval only when the query exceeds LLM's internal knowledge.
Outcome:	The proposed framework is superior to existing adaptive RAG methods on a diverse set of tasks.

Copied to clipboard

Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei

ActiView: Evaluating Active Perception Ability for Multimodal Large Language Models (2025.acl-long)

Challenge:	ELECTRA-style tasks are used to pretrain cross-lingual models for NLP tasks . masked language modeling tasks require massive computation resources, rendering such models quite expensive .
Approach:	They propose to use ELECTRA-style tasks to pre-train a cross-lingual language model . they propose to pretrain the model on multilingual and parallel corpora .
Outcome:	The proposed model outperforms baseline models on cross-lingual understanding tasks with much less computation cost.

Copied to clipboard

Ziyue Wang, Chi Chen, Fuwen Luo, Yurui Dong, Yuanchi Zhang, Yuzhuang Xu, Xiaolong Wang, Peng Li, Yang Liu

Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders (2021.emnlp-main)

Challenge:	Existing benchmarks for evaluating MLLMs have not addressed active perception . a novel benchmark is proposed to evaluate active perception in ML models .
Approach:	They propose a benchmark to evaluate active perception in Multimodal Large Language Models . they restrict the perceptual field of a model and require it to actively zoom or shift it .
Outcome:	The proposed benchmark focuses on a specialized form of Visual Question Answering (VQA) that eases and quantifies the evaluation yet challenging for existing MLLMs.

Copied to clipboard

Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

Manifold Learning-based Word Representation Refinement Incorporating Global and Local Information (2020.coling-main)

Challenge:	Existing work on improving cross-lingual transferability of NMT model is under-explored.
Approach:	They propose a model that leverages a multilingual pretrained encoder to improve cross-lingual transferability.
Outcome:	The proposed model outperforms mBART and m2m-100 on a zero-shot cross-lingual transfer task.

Copied to clipboard

Wenyu Zhao, Dong Zhou, Lin Li, Jinjun Chen

Expectation Confirmation Preference Optimization for Multi-Turn Conversational Recommendation Agent (2025.findings-acl)

Challenge:	Recent studies show word embedding models underestimate similarities between similar words and overestimate similarities between distant words.
Approach:	They propose two new word embedding methods that align original and re-fined embeddable spaces to a new refined semantic space.
Outcome:	The proposed methods outperform state-of-the-art methods for word representation refinement.

Copied to clipboard

Xueyang Feng, Jingsen Zhang, Jiakai Tang, Wei Li, Guohao Cai, Xu Chen, Quanyu Dai, Yue Zhu, Zhenhua Dong

Joint Multi-modal Aspect-Sentiment Analysis with Auxiliary Cross-modal Relation Detection (2021.emnlp-main)

Challenge:	Recent advances in Large Language Models (LLMs) have propelled the development of Conversational Recommendation Agents (CRAs).
Approach:	They propose a multi-turn preference optimization paradigm that leverages Expectation Confirmation Theory to explicitly model the evolution of user satisfaction throughout multi-turned dialogues.
Outcome:	The proposed paradigm eliminates the significant sampling overhead of existing MTPO methods while ensuring the optimization process drives meaningful improvements.

Copied to clipboard

Xincheng Ju, Dong Zhang, Rong Xiao, Junhui Li, Shoushan Li, Min Zhang, Guodong Zhou

How Pre-trained Language Models Capture Factual Knowledge? A Causal-Inspired Analysis (2022.findings-acl)

Challenge:	Existing studies on aspect-level sentiment analysis focus on extracting aspect terms and sentiment polarities separately.
Approach:	They propose a multi-modal joint learning approach with auxiliary cross-modal relation detection for multi-dimensional aspect-level sentiment analysis.
Outcome:	The proposed approach can obtain all aspect-level sentiment polarities dependent on the jointly extracted specific aspects.

Copied to clipboard

Shaobo Li, Xiaoguang Li, Lifeng Shang, Zhenhua Dong, Chengjie Sun, Bingquan Liu, Zhenzhou Ji, Xin Jiang, Qun Liu

Learning When to Translate for Streaming Speech (2022.acl-long)

Challenge:	Recent studies show that pre-trained language models can fill in the missing factual words in cloze-style prompts such as ”Dante was born in [MASK]” .
Approach:	They propose to quantitatively measure and evaluate the word-level patterns that PLMs depend on to generate the missing factual words.
Outcome:	The proposed model fills in the missing factual words in cloze-style prompts by relying on effective clues or shortcut patterns.

Copied to clipboard

Qian Dong, Yaoming Zhu, Mingxuan Wang, Lei Li

ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching (2026.findings-acl)

Challenge:	Existing methods waiting-and-translating for a fixed duration break speech acoustic units . Existing models waiting-for a set duration and generating partial sentences are not effective .
Approach:	They propose a monotonic segmentation module inside an encoder-decoder model to detect proper speech unit boundaries for a streaming speech input.
Outcome:	The proposed method outperforms existing methods on a speech translation dataset and achieves the best trade-off between translation quality and latency.

Copied to clipboard

Han Zhu, Wei Kang, Liyong Guo, Zengwei Yao, Fangjun Kuang, Weiji Zhuang, Zhaoqing Li, Zhifeng Han, Dong Zhang, Xin Zhang, Xingchen Song, Lingxuan Ye, Long Lin, Daniel Povey

Biology-Instructions: A Dataset and Benchmark for Multi-Omics Sequence Understanding Capability of Large Language Models (2025.findings-emnlp)

Challenge:	Existing autoregressive models for dialogue generation suffer from high latency and stability issues.
Approach:	They propose a non-autoregressive (NAR) zero-shot spoken dialogue generation model based on flow-matching.
Outcome:	The proposed model outperforms existing models in speech generation due to poor speech intelligibility and turn-taking precision.

Copied to clipboard

Haonan He, Yuchen Ren, Yining Tang, Ziyang Xu, Junxian Li, Minghao Yang, Di Zhang, Yuan Dong, Tao Chen, Shufei Zhang, Yuqiang Li, Nanqing Dong, Wanli Ouyang, Dongzhan Zhou, Peng Ye

DevEval: A Manually-Annotated Code Generation Benchmark Aligned with Real-World Code Repositories (2024.findings-acl)

Challenge:	Biology-Instructions is the first large-scale instruction-tuning dataset for multi-omics biological sequences.
Approach:	They propose a large-scale instruction-tuning dataset for multi-omics biological sequences . they propose 'chatMultiOmics' to overcome limitations of current LLMs on multi-ome tasks .
Outcome:	The proposed dataset bridges LLMs and complex biological sequence-related tasks while maintaining conversational fluency.

Copied to clipboard

Jia Li, Ge Li, Yunfei Zhao, Yongmin Li, Huanyu Liu, Hao Zhu, Lecheng Wang, Kaibo Liu, Zheng Fang, Lanshen Wang, Jiazheng Ding, Xuanming Zhang, Yuqi Zhu, Yihong Dong, Zhi Jin, Binhua Li, Fei Huang, Yongbin Li, Bin Gu, Mengfei Yang

TexSmart: A System for Enhanced Natural Language Understanding (2021.acl-demo)

Challenge:	Existing benchmarks are poorly aligned with real-world code repositories and are insufficient to evaluate the coding abilities of Large Language Models (LLMs).
Approach:	They propose a repository-level benchmark named DevEval to evaluate LLMs' coding abilities in real-world code repositories.
Outcome:	The proposed benchmarks show that the LLMs perform better in real-world code repositories than existing benchmarks.

Copied to clipboard

Lemao Liu, Haisong Zhang, Haiyun Jiang, Yangming Li, Enbo Zhao, Kun Xu, Linfeng Song, Suncong Zheng, Botong Zhou, Dick Zhu, Xiao Feng, Tao Chen, Tao Yang, Dong Yu, Feng Zhang, ZhanHui Kang, Shuming Shi

MINER: Multi-Interest Matching Network for News Recommendation (2022.findings-acl)

Challenge:	TexSmart supports fine-grained named entity recognition (NER) Large-scale fine-granular entity types are expected to provide richer semantic information for downstream NLP applications.
Approach:	They introduce TexSmart, a text understanding system that supports fine-grained named entity recognition (NER) and enhanced semantic analysis functionalities.
Outcome:	The proposed system supports fine-grained named entity recognition (NER) and enhanced semantic analysis functions.

Copied to clipboard

Jian Li, Jieming Zhu, Qiwei Bi, Guohao Cai, Lifeng Shang, Zhenhua Dong, Xin Jiang, Qun Liu

Representation Learning

Challenge:	Existing methods learn a single user embedding from user’s historical behaviors to represent the reading interest.
Approach:	They propose a poly attention scheme to learn multiple interest vectors for each user, which encodes the different aspects of user interest.
Outcome:	The proposed approach significantly outperforms existing state-of-the-art methods on the MIND news recommendation benchmark.

SelfRACG: Enabling LLMs to Self-Express and Retrieve for Code Generation (2025.emnlp-main)

Copied to clipboard

Qian Dong, Jia Chen, Qingyao Ai, Hongning Wang, Haitao Li, null Yiwu, Yao Hu, Yiqun Liu, Shaoping Ma

LearnAlign: Data Selection for LLM Reinforcement Learning with Improved Gradient Alignment (2026.findings-acl)

Challenge:	Existing retrieval-augmented code generation methods fail to accurately fetch the knowledge required for code generation for consecutive code fragments.
Approach:	They propose a paradigm that enables large language models to Self-express their information needs to enhance retrieval-augmented code generation methods.
Outcome:	Experiments show that SelfRACG can retrieve external knowledge that better aligns with the LLM’s own information needs, resulting in superior generation performance compared to vanilla RACG.

Copied to clipboard

Shipeng Li, Zhiqin Yang, Shikun Li, Xiaobo Xia, Hengyu Liu, Xinghua Zhang, Gaode Chen, Dong Fang, Ying Tai, Zhe Peng

CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding (2021.acl-long)

Challenge:	Reinforcement learning with verifiable rewards (RLVR) is a key technique for enhancing LLMs’ reasoning abilities, yet its data inefficiency remains a major bottleneck.
Approach:	They propose a gradient-alignment-based method which intelligently selects the learnable and representative training reasoning data for RLVR post-training.
Outcome:	Experiments on five reasoning benchmarks show that the proposed method significantly reduces training data requirements while improving performance.

Copied to clipboard

Dong Wang, Ning Ding, Piji Li, Haitao Zheng

DocOIE: A Document-level Context-Aware Dataset for OpenIE (2021.findings-acl)

Challenge:	Pre-trained language models are vulnerable to simple perturbations, causing poor robustness . recent studies show that adversarial training is useless or harmful for the model to detect these semantic changes.
Approach:	They propose to use adversarial training to improve the robustness of pre-trained models . they propose to construct negative examples with similar and opposite semantics .
Outcome:	Empirical results show that the proposed approach improves on sentiment analysis, reasoning, and reading comprehension tasks.

Copied to clipboard

Kuicai Dong, Zhao Yilin, Aixin Sun, Jung-Jae Kim, Xiaoli Li

Importance of Synthesizing High-quality Data for Text-to-SQL Parsing (2023.findings-acl)

Challenge:	Existing solutions focus on extracting tuples at sentence level, but sentences exist as part of a document rather than standalone.
Approach:	They propose to annotate 800 sentences from 80 documents to form a DocOIE dataset . they propose to use document-level context to improve OpenIE performance .
Outcome:	The proposed OpenIE model improves performance by incorporating documentlevel context into the dataset.

Copied to clipboard

Yiqun Hu, Yiyun Zhao, Jiarong Jiang, Wuwei Lan, Henghui Zhu, Anuj Chauhan, Alexander Hanbo Li, Lin Pan, Jun Wang, Chung-Wei Hang, Sheng Zhang, Jiang Guo, Mingwen Dong, Joseph Lilien, Patrick Ng, Zhiguo Wang, Vittorio Castelli, Bing Xiang

Revisiting and Advancing Chinese Natural Language Understanding with Accelerated Heterogeneous Knowledge Pre-training (2022.emnlp-industry)

Challenge:	Existing text-to-SQL parsers lack the data to perform well with augmented synthetic data.
Approach:	They propose a framework that imposes strong typing constraints and incorporates key relationships from schema.
Outcome:	The proposed framework improves on the high-quality synthesized SQL and natural language question (NLQ) models have significant accuracy boosts and achieve new state-of-the-art performance on spider.

Copied to clipboard

Taolin Zhang, Junwei Dong, Jianing Wang, Chengyu Wang, Ang Wang, Yinghui Liu, Jun Huang, Yong Li, Xiaofeng He

Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training (2021.emnlp-main)

Challenge:	Existing knowledge-enhanced pre-trained language models (KEPLMs) can capture internal knowledge, but can't understand external background knowledge.
Approach:	They propose to use Chinese knowledge-enhanced pre-trained language models to improve context-aware representations via learning from structured relations in knowledge bases.
Outcome:	Experiments show that Chinese knowledge-enhanced pre-trained language models outperform strong baselines over various benchmark NLP tasks and in different model sizes.

Copied to clipboard

Bo Zheng, Li Dong, Shaohan Huang, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

Out-of-Distribution Detection via LLM-Guided Outlier Generation for Text-attributed Graph (2025.findings-acl)

Challenge:	Existing models require a more expressive vocabulary to represent all languages . however, increasing the vocabulary size significantly slows down the pre-training speed .
Approach:	They propose an algorithm VoCap to determine the desired vocabulary capacity of each language.
Outcome:	The proposed algorithm improves cross-lingual model pre-training while reducing side effects of increasing vocabulary size.

Copied to clipboard

Xiangwei Lv, Mengze Li, Jingyuan Chen, Zhiang Dong, Sirui Han, Beishui Liao

Superpose Task-specific Features for Model Merging (2025.emnlp-main)

Challenge:	Text-Attributed Graphs (TAGs) are widely used in the real world.
Approach:	They propose to use Large Language Models to generate OOD-nodes with high quality . they also use LLMs to integrate existing nodes with LLM-generated edges .
Outcome:	The proposed method performs well on samples outside the In-Distribution (ID) data, but it is difficult to obtain high-quality OOD samples in the real world.

Copied to clipboard

Haiquan Qiu, You Wu, Dong Li, Jianmin Guo, Quanming Yao

Challenge:	Existing methods for model merging are limited by resource demands . recent studies validate the linear representation hypothesis .
Approach:	They propose a method that superposes task-specific features from individual models into a merged model.
Outcome:	The proposed method outperforms existing methods on multiple benchmarks and models.

Addressing Posterior Collapse with Mutual Information for Improved Variational Neural Machine Translation (2020.acl-main)

Copied to clipboard

Arya D. McCarthy, Xian Li, Jiatao Gu, Ning Dong

Machine Translation

Challenge:	Existing variational inference models ignore their latent variables, a phenomenon called posterior collapse.
Approach:	They propose a new loss function for conditional variational autoencoders that counteracts posterior collapse by using a modified evidence lower bound objective and a factorized decoder.
Outcome:	The proposed model yields improved translation quality compared to existing models on WMT RoEn and DeEn.

Can Monolingual Pretrained Models Help Cross-Lingual Classification? (2020.aacl-main)

Copied to clipboard

Zewen Chi, Li Dong, Furu Wei, Xianling Mao, Heyan Huang

Prompt Space Optimizing Few-shot Reasoning Success with Large Language Models (2024.findings-naacl)

Challenge:	Multilingual pretrained language models have shown impressive results for cross-lingual transfer, but due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors.
Approach:	They propose to transfer the knowledge from monolingual pretrained models to multilingual ones to improve zero-shot cross-lingual classification by using machine translation systems.
Outcome:	The proposed methods outperform vanilla multilingual fine-tuning on two cross-lingual classification benchmarks.

Copied to clipboard

Fobo Shi, Peijun Qing, Dong Yang, Nan Wang, Youbo Lei, Haonan Lu, Xiaodong Lin, Duantengchuan Li

MMedAgent: Learning to Use Medical Tools with Multi-modal Agent (2024.findings-emnlp)

Challenge:	Prompt engineering is an essential technique for enhancing the abilities of large language models (LLMs) by providing explicit and specific instructions.
Approach:	They propose a new approach that uses text embeddings to obtain basis vectors by matrix decomposition and constructs a space for representing all prompts.
Outcome:	The proposed approach significantly outperforms state-of-the-art prompt paradigms on ten public reasoning benchmarks.

Copied to clipboard

Binxu Li, Tiankai Yan, Yuanting Pan, Jie Luo, Ruiyang Ji, Jiayuan Ding, Zhe Xu, Shilong Liu, Haoyu Dong, Zihao Lin, Yixin Wang

VCB Bench: An Evaluation Benchmark for Audio-Grounded Large Language Model Conversational Agents (2026.findings-acl)

Challenge:	Multi-modal Large Language Models (MLLMs) exhibit limited generality and often fall short when compared to specialized models.
Approach:	They propose a multi-modal medical agent that picks the most suitable medical tools based on user inputs.
Outcome:	The proposed agent performs better than open-source models and the closed-source model, GPT-4o.

Copied to clipboard

Jiliang Hu, Wenfu Wang, Zuchao Li, Chenxing Li, Yiyang Zhao, Hanzhao Li, Liqiang Zhang, Meng Yu, Dong Yu

LLMTreeRec: Unleashing the Power of Large Language Models for Cold-Start Recommendations (2025.coling-main)

Challenge:	despite advances in multimodal conversational systems, current benchmarks lack comprehensive evaluation across key dimensions.
Approach:	They propose a Chinese benchmark built exclusively on real human speech to fill this gap . they assess LALMs across three complementary axes: instruction following, knowledge understanding, robustness .
Outcome:	VCB Bench assesses LALMs across three complementary axes: instruction following, knowledge understanding, and robustness . VCBM Bench provides reproducible and fine-grained framework for Chinese voice chat bots . results show significant performance disparities and offer tangible insights for future improvements .

Copied to clipboard

Wenlin Zhang, Chuhan Wu, Xiangyang Li, Yuhao Wang, Kuicai Dong, Yichao Wang, Xinyi Dai, Xiangyu Zhao, Huifeng Guo, Ruiming Tang

Distributed LLM Serving on Consumer-Grade GPUs by Reconciling Computation and Communication (2025.findings-emnlp)

Challenge:	Lack of training data leads to the system cold-start problem in recommendation systems, making them struggle to provide effective recommendations.
Approach:	They propose a tree-based LLM recommendation framework which structures all items into an item tree to improve the efficiency of LLM’s item retrieval.
Outcome:	The proposed framework outperforms the baseline model in the A/B test on Huawei industrial system.

Copied to clipboard

Lewei Jin, Kui Zhang, Yongqi Chen, null Zhuoyifan, Renjie Li, Yi Gao, Bowei Yang, Zhengong Cai, Wei Dong

Can We Edit Factual Knowledge by In-Context Learning? (2023.emnlp-main)

Challenge:	Large language models are reshaping internet services, and serving them is costly.
Approach:	They propose an efficient distributed LLM serving system that splits prefill and decode requests into smaller chunks .
Outcome:	The proposed system reduces TTFT, TPOT, and latency compared to the state-of-the-art system.

Copied to clipboard

Ce Zheng, Lei Li, Qingxiu Dong, Yuxuan Fan, Zhiyong Wu, Jingjing Xu, Baobao Chang

Better Highlighting: Creating Sub-Sentence Summary Highlights (2020.emnlp-main)

Challenge:	In-context knowledge editing (IKE) is a new paradigm for NLP research that can be applied to large language models with tens or hundreds of parameters.
Approach:	They propose to use in-context knowledge editing (IKE) without gradient updating to edit factual knowledge without a gradient update.
Outcome:	The proposed method achieves a competitive success rate compared to gradient-based methods on GPT-J but with fewer side effects.

Copied to clipboard

Sangwoo Cho, Kaiqiang Song, Chen Li, Dong Yu, Hassan Foroosh, Fei Liu

Summarization

Challenge:	Abstractive summarizations are considered to be less reliable because they distort the original meaning and can be confusing for readers.
Approach:	They propose a method to generate summary highlights that are understandable on their own to avoid confusion.
Outcome:	The proposed method allows summaries to be understood in context and avoids misdirecting readers to false conclusions.

Global Structure Knowledge-Guided Relation Extraction Method for Visually-Rich Document (2023.findings-emnlp)

Copied to clipboard

Xiangnan Chen, Qian Xiao, Juncheng Li, Duo Dong, Jun Lin, Xiaozhong Liu, Siliang Tang

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Existing methods focus on manipulating entity features to find pairwise relations, yet neglect the more fundamental structural information that links disparate entity pairs together.
Approach:	They propose a Visual Relation Extraction framework that generates relation predictions on entity pairs extracted from scanned images and incorporates global structural knowledge into the representations of the entities.
Outcome:	The proposed framework outperforms existing methods in fine-tuning setting and yields stronger data-efficient performance in the low-resource setting.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Uncertainty-Aware Test-Time Search for Optimization Problem Solving (2026.acl-long)

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Copied to clipboard

Linlin Yu, Xujiang Zhao, Dong Li, Yanchi Liu, Wei Cheng, Zhengzhang Chen, Chen Zhao, Feng Chen, Haifeng Chen

AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling (2024.acl-long)

Challenge:	Language model hallucinations and limited availability of labeled datasets often result in misaligned formulations, code errors and feasibility failures.
Approach:	They propose a Monte Carlo Tree Search framework that automates optimization problems from natural language descriptions with efficiency and reliability.
Outcome:	The proposed framework achieves state-of-the-art solution accuracy and reduces token usage.

Copied to clipboard

Jun Zhan, Junqi Dai, Jiasheng Ye, Yunhua Zhou, Dong Zhang, Zhigeng Liu, Xin Zhang, Ruibin Yuan, Ge Zhang, Linyang Li, Hang Yan, Jie Fu, Tao Gui, Tianxiang Sun, Yu-Gang Jiang, Xipeng Qiu

LeakDojo: Decoding the Leakage Threats of RAG Systems (2026.findings-acl)

Challenge:	Existing language models that use discrete representations for unified processing of various modalities are limited to text generation and do not include multimodal output.
Approach:	They propose a multimodal language model that utilizes discrete representations for unified processing of various modalities.
Outcome:	The proposed model can be trained stably without any alterations to existing models or training paradigms.

Copied to clipboard

Maosen Zhang, Jianshuo Dong, Lu Boting, Li Wenyue, Xiaoping Zhang, Tianwei Zhang, Han Qiu

A Knowledge Plug-and-Play Test Bed for Open-domain Dialogue Generation (2024.lrec-main)

Challenge:	Existing studies have failed to assess RAG leakage risks for large language models . constructing and maintaining highquality RAG knowledge databases has become increasingly costly .
Approach:	They propose a framework for controlled evaluation of RAG leakage using query generation and adversarial instructions.
Outcome:	The proposed framework compares six existing attacks across fourteen LLMs, four datasets, and diverse RAG systems.

Copied to clipboard

Xiangci Li, Linfeng Song, Lifeng Jin, Haitao Mi, Jessica Ouyang, Dong Yu

Clear Up Confusion: Advancing Cross-Domain Few-Shot Relation Extraction through Relation-Aware Prompt Learning (2024.naacl-short)

Challenge:	Knowledge-based open-domain dialogue generation aims to build chit-chat systems that talk to humans using mined support knowledge.
Approach:	They propose a benchmark for evaluating multi-source dialogue knowledge selection and response generation using Wikipedia's wizard of Wikipedia.
Outcome:	The proposed benchmark is called multi-source Wizard of Wikipedia (Ms.WoW) it contains clean support knowledge, grounded at the utterance level and partitioned into multiple knowledge sources.

Copied to clipboard

Ge Bai, Chenji Lu, Daichi Guo, Shilong Li, Ying Liu, Zhang Zhang, Guanting Dong, Ruifang Liu, Sun Yong

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs (2021.emnlp-main)

Challenge:	Existing approaches to few-shot Relation Extraction (RE) are prone to confusion when applying knowledge to a target domain with entirely new types of relations.
Approach:	They propose a relation-aware prompt learning method with pre-training to clear confusion by decomposing relation types through an innovative label prompt.
Outcome:	The proposed method outperforms previous sota methods and yields better results on cross-domain few-shot RE tasks.

Copied to clipboard

Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang, Saksham Singhal, Xian-Ling Mao, Heyan Huang, Xia Song, Furu Wei

PaperRegister: Boosting Flexible-grained Paper Search via Hierarchical Register Indexing (2026.acl-long)

Challenge:	Multilingual T5 pretrains a sequence-to-sequence model on monolingual texts, but it has shown promising results on many cross-lingual tasks.
Approach:	They propose a partially non-autoregressive objective for text-to-text pre-training and propose mT6 to improve cross-lingual transferability over multilingual T5.
Outcome:	The proposed model improves cross-lingual transferability over existing models.

Copied to clipboard

Zhuoqun Li, Xuanang Chen, Hongyu Lin, Yaojie Lu, Xianpei Han, Shanshan Jiang, Bin Dong, Le Sun

Humanity’s Last Code Exam: Can Advanced LLMs Conquer Human’s Hardest Code Competition? (2025.findings-emnlp)

Challenge:	Existing paper search systems lack detailed information to support finer-grained queries.
Approach:	They propose a paper-based index that transforms abstract-based corpus index into hierarchical index tree and offline can support paper search queries.
Outcome:	The proposed system achieves the SOTA performance and excels in fine-grained scenarios.

Copied to clipboard

Xiangyang Li, Xiaopeng Li, Kuicai Dong, null Zhangquanhu, Rongju Ruan, Xinyi Dai, Yasheng Wang, Ruiming Tang

DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation (2026.acl-long)

Challenge:	o4-mini(high) and Gemini-2.5 Pro achieve pass@1 rates of only 15.9% and 11.4%, respectively.
Approach:	They propose a harmonized online–offline sandbox that guarantees fully reproducible evaluation.
Outcome:	The proposed test reflects the advanced reasoning and code generation ability of large language models.

Copied to clipboard

Li Huang, Zhongxin Liu, Yifan Wu, Tao Yin, Dong li, Jichao Bi, Nankun Mu, Hongyu Zhang, Meng Yan

Don’t Just Listen, Try Planning: Graph-based Retrieval-Generation Agent for Long-form Audio Meeting Understanding (2026.findings-acl)

Challenge:	Large Language Models (LLMs) can replicate insecure patterns from training data.
Approach:	They propose a framework that leverages distributed security-relevant cues by aggregating representations from multiple upper layers via an attention-based module.
Outcome:	Experiments show that the framework improves the secure-and-correct generation rate by 11.9% over baselines.

Copied to clipboard

Quanwei Tang, Dong Zhang, Shoushan Li, Guodong Zhou

GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators (2024.acl-long)

Challenge:	Existing question answering (QA) datasets for long audio meetings suffer from acoustic information loss and poor long-term dependency capture.
Approach:	They propose a question answering dataset that captures three core dimensions of long-form audio meeting content.
Outcome:	The proposed model captures three core dimensions of long-form audio meeting content: complex semantics, multi-speaker interactions, and quite long timestamps.

Copied to clipboard

Yuchen Hu, Chen Chen, Chao-Han Yang, Ruizhe Li, Dong Zhang, Zhehuai Chen, EngSiong Chng

Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network (P18-1)

Challenge:	Recent advances in large language models (LLMs) have stepped forward the development of multilingual speech and machine translation by its reduced representation errors and incorporated external knowledge.
Approach:	They propose a generative paradigm for translation tasks that integrates the diverse translation versions in N-best list.
Outcome:	The proposed model outperforms the state-of-the-art model on speech and machine translation benchmarks on various languages.

Copied to clipboard

Xiangyang Zhou, Lu Li, Daxiang Dong, Yi Liu, Ying Chen, Wayne Xin Zhao, Dianhai Yu, Hua Wu

Tunable Soft Prompts are Messengers in Federated Learning (2023.findings-emnlp)

Challenge:	Existing models for matching dialogue responses rely on semantic and functional dependencies . a recent study only uses the last utterance in context for matching a reply .
Approach:	They propose a model that matches a response with its multi-turn context using attention.
Outcome:	The proposed model outperforms the state-of-the-art models on two large-scale multi-turn response selection tasks.

Copied to clipboard

Chenhe Dong, Yuexiang Xie, Bolin Ding, Ying Shen, Yaliang Li

Interpretable and Low-Resource Entity Matching via Decoupling Feature Learning from Decision Making (2021.acl-long)

Challenge:	Existing methods to protect model privacy in federated learning (FL) are limited.
Approach:	They propose a federated learning approach that provides model privacy protection via tunable soft prompts.
Outcome:	The proposed approach provides protection for the global model while reducing communication and computation costs.

Copied to clipboard

Zijun Yao, Chengjiang Li, Tiansi Dong, Xin Lv, Jifan Yu, Lei Hou, Juanzi Li, Yichi Zhang, Zelin Dai

Learning to Ask Unanswerable Questions for Machine Reading Comprehension (P19-1)

Challenge:	Entity Matching (EM) aims at recognizing entity records that denote the same real-world object.
Approach:	They propose a novel EM framework that consists of Heterogeneous Information Fusion and Key Attribute Tree Induction to decouple feature representation from matching decision.
Outcome:	The proposed framework outperforms SOTA EM models on 6 public datasets and 3 industrial datasets.

Copied to clipboard

Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

CLIP Models are Few-Shot Learners: Empirical Studies on VQA and Visual Entailment (2022.acl-long)

Challenge:	Existing models for extractive reading comprehension are not good at deciding whether no answer is presented in the context.
Approach:	They propose a data augmentation technique by automatically generating relevant unanswerable questions according to an answerable question paired with its corresponding paragraph that contains the answer.
Outcome:	The proposed model performs better on the SQuAD 2.0 dataset than the baseline model and the BERT-large model.

Copied to clipboard

Haoyu Song, Li Dong, Weinan Zhang, Ting Liu, Furu Wei

Visual Data in NLP

Challenge:	Previously, CLIP was only regarded as a powerful visual encoder.
Approach:	They propose a parameter-efficient fine-tuning strategy to boost CLIP's few-shot performance on a visual entailment task without introducing any additional pre-training procedure.
Outcome:	The proposed strategy achieves competitive zero/few-shot results on visual question answering and visual entailment tasks without introducing any additional pre-training procedure.

Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack (2024.acl-long)

Copied to clipboard

Yu Fu, Yufei Li, Wen Xiao, Cong Liu, Yue Dong

DisastQA: A Comprehensive Benchmark for Evaluating Question Answering in Disaster Management (2026.findings-acl)

Challenge:	Recent developments in balancing usefulness and safety of large language models raise a critical question . current attacks, especially adversarial ones that manipulate malicious prompts, often aim to manipulate the input .
Approach:	They show that LLMs can effectively summarize malicious long documents but often refuse to translate them.
Outcome:	The findings highlight a vulnerability in LLMs that can't translate or summarize documents . the study focuses on LLM models, Gemini and GPT-4, which can' be exploited .

Copied to clipboard

Zhitong Chen, Kai Yin, Xiangjue Dong, Chengkai Liu, Xiangpeng Li, Bo Li, Junwei Ma, Yiming Xiao, Ali Mostafavi, James Caverlee

Maximum Score Routing For Mixture-of-Experts (2025.findings-acl)

Challenge:	Existing benchmarks for question answering (QA) are lacking in a high-stakes environment.
Approach:	They propose a rigorously verified benchmark of 3,000 expert-annotated questions . they propose 'keypoint-based evaluation protocol' emphasizing factual completeness over verbosity .
Outcome:	Experiments with 20 models reveal substantial divergences from general-purpose models such as MMLU-Pro.

Copied to clipboard

Bowen Dong, Yilong Fan, Yutao Sun, Zhenyu Li, Tengyu Pan, Zhou Xun, Jianyong Wang

PACE: Improving Prompt with Actor-Critic Editing for Large Language Model (2024.findings-acl)

Challenge:	Traditional mixture-of-experts (MoE) networks impose an expert capacity constraint to ensure GPU-friendly computation.
Approach:	They propose a routing paradigm that dynamically allocates input tokens to top-k experts through differentiable sparse transformations, enabling scalable model capacity while preserving computational efficiency.
Outcome:	The proposed model achieves lower training losses and higher evaluation scores at equivalent FLOPs compared to constrained and unconstrained baselines.

Copied to clipboard

Yihong Dong, Kangcheng Luo, Xue Jiang, Zhi Jin, Ge Li

StorySparkQA: Expert-Annotated QA Pairs with Real-World Knowledge for Children’s Story-Based Learning (2024.emnlp-main)

Challenge:	Prompt with Actor-Critic Editing (PACE) for LLMs improves performance of different human-written prompts, resulting in significant performance discrepancies.
Approach:	They propose to use LLMs as actors and critics to enable automatic prompt editing by taking feedback from both actors performing prompt and criticizing response into account.
Outcome:	The proposed model improves the performance of human-written prompts by 98% and compares to high-quality human-writing prompts.

Copied to clipboard

Jiaju Chen, Yuxuan Lu, Shao Zhang, Bingsheng Yao, Yuanzhe Dong, Ying Xu, Yunyao Li, Qianwen Wang, Dakuo Wang, Yuling Sun

SeqXGPT: Sentence-Level AI-Generated Text Detection (2023.emnlp-main)

Challenge:	Existing story reading systems fail to capture the nuances of how education experts think when conducting interactive story reading activities.
Approach:	They propose to use existing question-answering (QA) datasets to capture experts' annotations and thinking process to construct a story-based annotation framework.
Outcome:	The proposed framework captures experts’ annotations and thinking process and can be used to generate 5, 868 expert-annotated QA pairs with real-world knowledge.

Copied to clipboard

Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, Xipeng Qiu

GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator (2023.acl-long)

Challenge:	Existing methods for sentence-level AIGT detection are weak . large language models (LLMs) can generate human-like content .
Approach:	They propose a sentence-level AIGT detection challenge using LLMs as log probability lists . they propose 'check' GPT' method that uses log probability list features to detect AIGT .
Outcome:	The proposed method surpasses baseline methods in sentence- and document-level detection challenges.

Copied to clipboard

Jian Yang, Shuming Ma, Li Dong, Shaohan Huang, Haoyang Huang, Yuwei Yin, Dongdong Zhang, Liqun Yang, Furu Wei, Zhoujun Li

ImageNetVC: Zero- and Few-Shot Visual Commonsense Evaluation on 1000 ImageNet Categories (2023.findings-emnlp)

Challenge:	Existing pre-training methods underutilize the benefits of language understanding for generation.
Approach:	They propose a GAN-style model for encoder-decoder pre-training with an auxiliary discriminator.
Outcome:	The proposed model outperforms existing pre-trained models and achieves state-of-the-art performance.

Copied to clipboard

Heming Xia, Qingxiu Dong, Lei Li, Jingjing Xu, Tianyu Liu, Ziwei Qin, Zhifang Sui

WESR: A Benchmark and Strong Baseline for Word-level Event-Speech Recognition (2026.findings-acl)

Challenge:	Large Language Models (LLMs) are becoming general-purpose APIs, requiring visual knowledge to be understood.
Approach:	They propose to evaluate the visual capability of large-scale large-language models through visual commonsense evaluation using a human-annotated dataset.
Outcome:	The proposed dataset compares the visual commonsense knowledge of large-scale models with those of unimodal LLMs and visually augmented models.

Copied to clipboard

Chenchen Yang, Kexin Huang, Liwei Fan, Qian Tu, Botian Jiang, Dong Zhang, Linqi Yin, Shimin Li, Zhaoye Fei, Qinyuan Cheng, Xipeng Qiu

Large Language Models as Zero-shot Dialogue State Tracker through Function Calling (2024.acl-long)

Challenge:	aaron carroll: the precise localization of non-verbal vocal events remains a critical yet under-explored challenge. carroll says current methods suffer from insufficient task definitions with limited category coverage. carrol: knowing exactly where an event occurred is not enough; knowing exactly what it happened is.
Approach:	They propose a taxonomy of 21 vocal events with a new categorization into discrete versus continuous types.
Outcome:	The proposed model disentangles ASR errors from event detection while maintaining ASR quality.

Copied to clipboard

Zekun Li, Zhiyu Chen, Mike Ross, Patrick Huber, Seungwhan Moon, Zhaojiang Lin, Xin Dong, Adithya Sagar, Xifeng Yan, Paul Crook

SceneGenAgent: Precise Industrial Scene Generation with Coding Agent (2025.acl-long)

Challenge:	Large language models (LLMs) are increasingly prevalent in conversational systems due to their advanced understanding and generative capabilities in general contexts.
Approach:	They propose a method for solving dialogue state tracking (DST) with large language models through function calling.
Outcome:	The proposed approach improves zero-shot DST, allowing adaptation to diverse domains without extensive data collection or model tuning.

Copied to clipboard

Xiao Xia, Dan Zhang, Zibo Liao, Zhenyu Hou, Tianrui Sun, Jing Li, Ling Fu, Yuxiao Dong

Compressing Context to Enhance Inference Efficiency of Large Language Models (2023.emnlp-main)

Challenge:	Recent work on scene generation focuses on generating 3D scenes from textual descriptions . however, the task of generating industrial scenes with LLMs is complex and requires precise measurements and positioning .
Approach:	They propose an LLM-based agent for generating industrial scenes through C# code.
Outcome:	Experiments show that LLMs powered by SceneGenAgent exceed their original performance . the agent achieves 81.0% success rate in real-world industrial scene generation tasks .

Copied to clipboard

Yucheng Li, Bo Dong, Frank Guerin, Chenghua Lin

Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension (D19-58)

Challenge:	Large language models (LLMs) have demonstrated remarkable power and impressive generalisation abilities across various tasks.
Approach:	They propose a method that prunes redundancies in the input context to make the input more compact.
Outcome:	The proposed method reduces memory and inference time while maintaining comparable performance compared to full context.

Copied to clipboard

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Lei Cui, Songhao Piao, Ming Zhou

Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference (2025.naacl-long)

Challenge:	Experimental results show that unified model outperforms other models that treat encoding and matching separately.
Approach:	They evaluate a unified model with Transformer layers for machine reading comprehension . they find that the model learns different modeling strategies compared with previous models .
Outcome:	The unified model outperforms models with Transformer layers on the machine reading comprehension task.

Copied to clipboard

Zeping Li, Xinlong Yang, Ziheng Gao, Ji Liu, Guanchen Li, Zhuang Liu, Dong Li, Jinzhang Peng, Lu Tian, Emad Barsoum

Machine Translation

Challenge:	Existing methods such as Medusa lack adequate information interaction between different drafting heads.
Approach:	They propose an enhanced speculative decoding framework that builds upon Medusa and integrates a drafting block capable of parallel inference.
Outcome:	The proposed framework outperforms Medusa in terms of head accuracy and latency.

LongReward: Improving Long-context Large Language Models with AI Feedback (2025.acl-long)

Copied to clipboard

Jiajie Zhang, Zhongni Hou, Xin Lv, Shulin Cao, Zhenyu Hou, Yilin Niu, Lei Hou, Yuxiao Dong, Ling Feng, Juanzi Li

A Copy-Augmented Generative Model for Open-Domain Question Answering (2022.acl-short)

Challenge:	In recent years, significant advancements have been achieved in the development of long-context large language models (LLMs).
Approach:	They propose a method that utilizes an off-the-shelf LLM to provide rewards for long-context model responses from four human-valued dimensions: helpfulness, logicality, faithfulness, and completeness.
Outcome:	The proposed method improves models’ long-context performance and enhances their ability to follow short instructions.

Copied to clipboard

Shuang Liu, Dong Wang, Xiaoguang Li, Minghui Huang, Meizhen Ding

TabularMath: Understanding Math Reasoning over Tables with Large Language Models (2026.findings-acl)

Challenge:	Existing open-domain question answering approaches follow a two-stage paradigm retriever then reader.
Approach:	They propose a novel reader-based generative approach that incorporates extractive and generative readers.
Outcome:	The proposed model improves on two benchmark datasets, Natural Questions and TriviaQA.

Copied to clipboard

Shi-Yu Tian, Zhi Zhou, Wei Dong, Kun-Yang Yu, Ming Yang, Zi-Jian Cheng, Lan-Zhe Guo, Yu-Feng Li

Rectified Sparse Attention for Efficient Long-Sequence Generation (2026.findings-acl)

Challenge:	Mathematical reasoning has long been a key benchmark for evaluating large language models.
Approach:	They propose a framework that transforms math word problems into scalable tabular reasoning tasks.
Outcome:	The proposed framework transforms math word problems into scalable and verified tabular reasoning tasks.

Copied to clipboard

Yutao Sun, Tianzhu Ye, Li Dong, Yuqing Xia, Jian Chen, Yizhao Gao, Shijie Cao, Jianyong Wang, Furu Wei

SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoning (2026.findings-acl)

Challenge:	Recent sparse decoding methods improve efficiency but suffer from KV cache misalignment, resulting in performance degradation.
Approach:	They propose a method that combines block-sparse attention with periodic dense rectification to bound error accumulation and preserve alignment with the pretraining distribution.
Outcome:	Experiments on math reasoning, language modeling, and retrieval tasks show that ReSA achieves near-lossless generation quality with significantly improved efficiency.

Copied to clipboard

Zheng Li, Qingxiu Dong, Jingyuan Ma, Di Zhang, Kai Jia, Zhifang Sui

Crossing the Reward Bridge: Expanding Reinforcement Learning with Verifiable Rewards Across Diverse Domains (2026.acl-long)

Challenge:	Recent large reasoning models have shown exceptional performance on various tasks, but they consume excessive tokens even for simple queries, leading to resource waste and prolonged user latency.
Approach:	They propose a self-adaptive reasoning strategy that automatically allocates budgets according to problem complexity and introduces GRPO for reinforcement learning to reduce output length.
Outcome:	The proposed model achieves an average response length compression of 61% on math reasoning tasks while maintaining accuracy.

Copied to clipboard

Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, Dong Yu

Challenge:	Reinforcement learning with verifiable rewards (RLVR) has been effective on structured tasks, but its reliance on simple, rule-based verifiers creates a bottleneck.
Approach:	They propose a framework that uses a generative verifier to provide soft, probabilistic rewards.
Outcome:	The proposed framework outperforms existing models up to 10x their size and can be scalable and effective.

How Can Cross-lingual Knowledge Contribute Better to Fine-Grained Entity Typing? (2022.findings-acl)

Copied to clipboard

Hailong Jin, Tiansi Dong, Lei Hou, Juanzi Li, Hui Chen, Zelin Dai, Qu Yincen

ReflectEvo: Improving Meta Introspection of Small LLMs by Learning Self-Reflection (2025.findings-acl)

Challenge:	Extensive experiments on multi-lingual datasets show that our method significantly outperforms multiple baselines and can robustly handle negative transfer.
Approach:	They propose to transfer semantic knowledge from rich-resourced languages to low-resource languages by using multilingual transfer learning.
Outcome:	The proposed model outperforms baselines and can handle negative transfer.

Copied to clipboard

Jiaqi Li, Xinyi Dong, Yang Liu, Zhizhuo Yang, Quansen Wang, Xiaobo Wang, Song-Chun Zhu, Zixia Jia, Zilong Zheng

MetaPrompting: Learning to Learn Better Prompts (2022.coling-1)

Challenge:	ReflectEvo-460k is a large-scale, comprehensive, self-generated reflection dataset with broadened instructions and diverse multi-domain tasks.
Approach:	They propose a pipeline that iteratively generates self-reflection for self-training and a large-scale reflection dataset with broadened instructions and diverse multi-domain tasks.
Outcome:	The proposed pipeline improves Llama-3 reasoning ability by up to 71.2% and Mistral by upto 44.4%.

Copied to clipboard

Yutai Hou, Hongyuan Dong, Xinghao Wang, Bohan Li, Wanxiang Che

Beyond Single Frames: Can LMMs Comprehend Implicit Narratives in Comic Strip? (2025.findings-emnlp)

Challenge:	Recent research on prompting moves from discrete tokens based "hard prompts" to continuous "soft prompts", which employ learnable vectors as pseudo prompt tokens and achieve better performance.
Approach:	They propose a generalized soft prompting method that uses model-agnostic meta-learning to find better initialization for soft prompts.
Outcome:	The proposed method improves on three datasets and brings new state-of-the-art performance.

Copied to clipboard

Xiaochen Wang, Heming Xia, Jialin Song, Longyu Guan, Qingxiu Dong, Rui Li, Yixin Yang, Yifan Pu, Weiyao Luo, Yiru Wang, Xiangdi Meng, Wenjie Li, Zhifang Sui

Exploring the Efficacy of Automatically Generated Counterfactuals for Sentiment Analysis (2021.acl-long)

Challenge:	Large Multimodal Models have demonstrated strong performance on vision-language benchmarks, yet current evaluations focus on single-image reasoning.
Approach:	STRIPCIPHER is a benchmark designed to evaluate model ability on understanding implicit narratives in silent comics.
Outcome:	STRIPCIPHER is a high-quality, human-annotated dataset featuring fine-grained annotations and comprehensive coverage of varying difficulty levels.

Copied to clipboard

Linyi Yang, Jiazheng Li, Padraig Cunningham, Yue Zhang, Barry Smyth, Ruihai Dong

LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks (2025.acl-long)

Challenge:	Existing approaches to improve performance of deep neural models are limited by the nature of spurious patterns in the data.
Approach:	They propose to use augmented data to generate spurious patterns in NLP models . they propose to generate counterfactual data for data augmentation and explanation .
Outcome:	The proposed approach improves performance on augmented data and on human-generated data.

Copied to clipboard

Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

VideoPro: Adaptive Program Reasoning for Long Video Understanding (2026.acl-long)

Challenge:	Existing benchmarks on longcontext large language models fail to reflect their deep understanding capabilities across diverse tasks.
Approach:	They propose a benchmark to assess the ability of long-context large language models to handle long-text problems.
Outcome:	The proposed model achieves 50.1% accuracy when directly answering the questions . human experts achieve only 53.7% accuracy under a 15-minute time constraint .

Copied to clipboard

Chenglin Li, Feng Han, Yikun Wang, Ruilin Li, Shuai Dong, Haowen Hou, Haitao Li, Qianglong Chen, Feng Tao, Jingqi Tong, Yin Zhang, Jiaqi Wang

Hierarchical Document Refinement for Long-context Retrieval-augmented Generation (2025.acl-long)

Challenge:	Existing methods for understanding long videos are limited due to the sparsity of visual evidence relevant to a given query.
Approach:	They propose a framework that enables VideoLLMs to reason over long videos and refine their predictions through executable programs.
Outcome:	The proposed framework outperforms existing methods across long-video understanding benchmarks.

Copied to clipboard

Jiajie Jin, Xiaoxi Li, Guanting Dong, Yuyao Zhang, Yutao Zhu, Yongkang Wu, Zhonghua Li, Ye Qi, Zhicheng Dou

Knowledge Neurons in Pretrained Transformers (2022.acl-long)

Challenge:	Real-world RAG applications often encounter long-context input scenarios where redundant information and noise results in higher inference costs and reduced performance.
Approach:	They propose an efficient plug-and-play refiner that leverages the structural characteristics of long documents.
Outcome:	Experiments on seven QA datasets show that LongRefiner achieves competitive performance in various scenarios while using 10x fewer computational costs and latency compared to baseline.

Copied to clipboard

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, Furu Wei

AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time (2025.emnlp-main)

Challenge:	Existing studies show that pretrained language models are good at recalling factual knowledge without fine-tuning.
Approach:	They propose a method to identify neurons that express factual knowledge in pretrained Transformers by filling-in-the-blank cloze queries.
Outcome:	The proposed method can be used to edit, erase, and update factual knowledge without fine-tuning.

Copied to clipboard

Junyu Zhang, Runpei Dong, Han Wang, Xuying Ning, Haoran Geng, Peihao Li, Xialin He, Yutong Bai, Jitendra Malik, Saurabh Gupta, Huan Zhang

SnapNTell: Enhancing Entity-Centric Visual Question Answering with Retrieval Augmented Multimodal LLM (2024.findings-emnlp)

Challenge:	Existing monotonic scaling methods for large reasoning models are not reliable.
Approach:	They propose a universal framework for modulating reasoning progress in large reasoning models at test time.
Outcome:	The proposed framework unifies and generalizes existing monotonic scaling methods and enables flexible and dense slow-to-fast reasoning modulation.

Copied to clipboard

Jielin Qiu, Andrea Madotto, Zhaojiang Lin, Paul Crook, Yifan Xu, Babak Damavandi, Xin Dong, Christos Faloutsos, Lei Li, Seungwhan Moon

Theory-optimal Quantization Based on Flatness (2026.acl-long)

Challenge:	Vision-extended LLMs have made significant strides in VQA, but they still encounter significant difficulties in handling queries involving long-tail entities.
Approach:	They propose a benchmark to test models' ability to identify entities and provide detailed, entity-specific knowledge by combining 10 images and 10 knowledge-intensive QA pairs.
Outcome:	The proposed model outperforms existing methods on the SnapNTell dataset, achieving a 66.5% improvement in the BELURT score.

Copied to clipboard

Xiusheng Huang, Zhe Li, Xuanwu Yin, Lu Wang, Yequan Wang, Dong Li, Emad Barsoum, Kang Liu

Linguistic Theories

Challenge:	Recent approaches to quantization of Large Language Models (LLMs) have been widely adopted due to activation outliers, which degrade model performance especially at lower bit precision.
Approach:	They propose a new metric for quantization that strategically distributes outlier magnitudes across matrix dimensions via optimized diagonal operations.
Outcome:	The proposed framework achieves less than 1% accuracy drop in W4A4 quantization on the LLaMA-3-8B model and reduces the performance gap by 39.1% on the more challenging W2A4KV16 model.

TermDiffuSum: A Term-guided Diffusion Model for Extractive Summarization of Legal Documents (2025.coling-main)

Copied to clipboard

Xiangyun Dong, Wei Li, Yuquan Le, Zhangyue Jiang, Junxi Zhong, Zhong Wang

CoIR: A Comprehensive Benchmark for Code Information Retrieval Models (2025.acl-long)

Challenge:	Recent studies have explored diffusion models for extractive summarization task, showcasing their remarkable capabilities.
Approach:	They propose a term-guided diffusion model for extractive summarization of legal documents that incorporates legal terminology into the model via a well-designed multifactor fusion noise weighting schedule.
Outcome:	The proposed model outperforms existing models on a self-constructed legal summarization dataset and achieves improvements of 3.10, 2.84, and 2.89 on three public datasets.

Copied to clipboard

Xiangyang Li, Kuicai Dong, Yi Quan Lee, Wei Xia, Hao Zhang, Xinyi Dai, Yasheng Wang, Ruiming Tang

PSSAT: A Perturbed Semantic Structure Awareness Transferring Method for Perturbation-Robust Slot Filling (2022.coling-1)

Challenge:	Existing methods and benchmarks for information retrieval are inadequately representing the diversity of code in various domains and tasks.
Approach:	They propose a benchmark specifically designed to assess code retrieval capabilities.
Outcome:	The proposed benchmark aims to invigorate research in the code retrieval domain . it shares the same data schema as other popular benchmarks like MTEB and BEIR .

Copied to clipboard

Guanting Dong, Daichi Guo, Liwen Wang, Xuefeng Li, Zechen Wang, Chen Zeng, Keqing He, Jinzheng Zhao, Hao Lei, Xinyue Cui, Yi Huang, Junlan Feng, Weiran Xu

Enhancing Partially Relevant Video Retrieval with Robust Alignment Learning (2025.findings-emnlp)

Challenge:	Existing slot filling models memorize inherent patterns of entities and contexts from training data.
Approach:	They propose a perturbed semantic structure awareness transferring method for slot filling models . they use two MLM-based training strategies to learn contextual semantic structure and word distribution .
Outcome:	The proposed method outperforms existing methods and gains strong generalization while preventing model from memorizing inherent patterns of entities and contexts.

Copied to clipboard

Long Zhang, Peipei Song, Jianfeng Dong, Kun Li, Xun Yang

CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)

Challenge:	Existing methods focus on enhancing multi-scale clip representations but lack robust data alignment . inherent data uncertainty renders PRVR vulnerable to distractor videos with spurious similarities .
Approach:	proposed framework for partially relevant video retrieval aims to retrieve untrimmed videos partially relevant to a given query.
Outcome:	The proposed framework can be seamlessly integrated into existing architectures.

Copied to clipboard

Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan

Challenge:	Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages .
Approach:	They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models .
Outcome:	The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English .

Type Enhanced BERT for Correcting NER Errors (2023.findings-acl)

Copied to clipboard

Kuai Li, Chen Chen, Tao Yang, Tianming Du, Peijie Yu, Dong Du, Feng Zhang

InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance (2024.emnlp-main)

Challenge:	Named entity recognition (NER) is the task of identifying spans that belong to particular categories, such as person, location, organization, etc.
Approach:	They propose a method that integrates named entity’s type information into BERT by an adapter layer and integrates it into a gazetteer.
Outcome:	The proposed method outperforms baselines in multiple corpus.

Copied to clipboard

Pengyu Wang, Dong Zhang, Linyang Li, Chenkun Tan, Xinghao Wang, Mozhi Zhang, Ke Ren, Botian Jiang, Xipeng Qiu

RESIN: A Dockerized Schema-Guided Cross-document Cross-lingual Cross-media Information Extraction and Event Tracking System (2021.naacl-demos)

Challenge:	Existing methods for enhancing harmlessness and helpfulness of large language models (LLMs) involve complex and resource-intensive training processes.
Approach:	They propose a method that decouples harmlessness from helpfulness during inference phase.
Outcome:	The proposed method significantly reduces the attack success rate (ASR) of harmful instructions and jailbreak instructions while maintaining almost unchanged performance in downstream tasks.

Copied to clipboard

Haoyang Wen, Ying Lin, Tuan Lai, Xiaoman Pan, Sha Li, Xudong Lin, Ben Zhou, Manling Li, Haoyu Wang, Hongming Zhang, Xiaodong Yu, Alexander Dong, Zhenhailong Wang, Yi Fung, Piyush Mishra, Qing Lyu, Dídac Surís, Brian Chen, Susan Windisch Brown, Martha Palmer, Chris Callison-Burch, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Heng Ji

Visual Data in NLP

Challenge:	We present a new information extraction system that can construct temporal event graphs from news documents.
Approach:	They propose a temporal event graph extraction system that can extract news documents . they extend the system from sentence-level event extraction to cross-document cross-media event extraction .
Outcome:	The proposed system can extract temporal event graphs from news documents in multiple languages and multiple data modalities.

Trajectory2Task: Training Robust Tool-Calling Agents with Synthesized Yet Verifiable Data for Complex User Intents (2026.acl-long)

Copied to clipboard

Ziyi Wang, Yuxuan Lu, Yimeng Zhang, Pei Chen, Ziwei Dong, Jing Huang, Jiri Gesi, Xianfeng Tang, Chen Luo, Qun Liu, Yisi Sang, Hanqing Lu, Manling Li, Jin Lai, Dakuo Wang

Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System (2025.acl-long)

Challenge:	Tool-calling agents are increasingly deployed in real-world customer-facing workflows . but most studies on tool-callers focus on idealized settings with general, fixed, and well-specified tasks.
Approach:	They propose a tool-calling agent-based data pipeline that converts trajectories into user-facing tasks with controlled intent adaptations.
Outcome:	The proposed pipeline can be used to study tool use under three scenarios.

Copied to clipboard

Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, Nanqing Dong

Discovering Semantic Subdimensions through Disentangled Conceptual Representations (2025.findings-emnlp)

Challenge:	Recent AI methods have shown promise in tasks such as hypothesis generation and experimental design, but they fail to replicate the collaborative nature of real-world scientific practices.
Approach:	They propose a virtual scientific system that mimics the collaborative nature of scientific research by organizing a team of agents to generate, evaluate, and refine research ideas.
Outcome:	The proposed system outperforms the state-of-the-art method in producing new scientific ideas and offers valuable insights to guide future research.

Copied to clipboard

Yunhao Zhang, Shaonan Wang, Nan Lin, Xinyi Dong, Chong Li, Chengqing Zong

Explainability & Interpretability in NLP

Challenge:	Existing approaches focus on predefined dimensions that overlook finer conceptual distinctions . a new framework is proposed to investigate the subdimensions underlying coarse-grained semantic dimensions .
Approach:	They propose a framework that decomposes word embeddings into multiple sub-embeddings . they propose to map these subdimensions to brain activation to assess their plausibility .
Outcome:	The proposed framework decomposes word embeddings from large language models into sub-embeddings, each encoding specific semantic information.

SURE: Safety Understanding and Reasoning Enhancement for Multimodal Large Language Models (2025.emnlp-main)

Copied to clipboard

Yuxin Gou, Xiaoning Dong, Qin Li, Shishen Gu, Richang Hong, Wenbo Hu

EDDA: An Encoder-Decoder Data Augmentation Framework for Zero-Shot Stance Detection (2024.lrec-main)

Challenge:	Existing multimodal large language models incorporate visual and textual information, but introduces new and complex safety risks.
Approach:	They propose a safety reasoning framework that integrates visual modalities into multimodal models to help them resist jailbreak attacks.
Outcome:	The proposed framework improves model safety while avoiding over-defense . it is based on a large-scale safety reasoning dataset .

Copied to clipboard

Daijun Ding, Li Dong, Zhichao Huang, Guangning Xu, Xu Huang, Bo Liu, Liwen Jing, Bowen Zhang

XY-Tokenizer: Mitigating the Semantic-Acoustic Conflict in Low-Bitrate Speech Codecs (2026.acl-long)

Challenge:	Existing methods for enhancing text or data are limited by lack of logical connections between generated texts and training data.
Approach:	They propose an encoder-decoder data augmentation framework that combines large language models and chain-of-thought prompting to summarize texts into target-specific if-then rationales, establishing logical relationships.
Outcome:	The proposed framework significantly improves over state-of-the-art methods on benchmark datasets while enabling interpretable rationale-based learning.

Copied to clipboard

Yitian Gong, Luozhijie Jin, Kuangwei Chen, Dong Zhang, Ruifan Deng, Xiaogui Yang, Xin Zhang, Zhaoye Fei, Qinyuan Cheng, Shimin Li, Xipeng Qiu

Tree-of-Code: A Self-Growing Tree Framework for End-to-End Code Generation and Execution in Complex Tasks (2025.findings-acl)

Challenge:	Existing speech codecs struggle to balance these objectives at low bitrates . XY-Tokenizer achieves stronger semantic alignment than representative semantic-distillation codec .
Approach:	They propose a low-bitrate speech codec that aligns discrete speech representations with text while preserving fine-grained acoustic details for reconstruction.
Outcome:	The proposed codec outperforms existing low-bitrate speech codecs in speech understanding and generation tasks.

Copied to clipboard

Ziyi Ni, Yifan Li, Ning Yang, Dou Shen, Pin Lyu, Daxiang Dong

Adversarial Attacks on Parts of Speech: An Empirical Study in Text-to-Image Generation (2024.findings-emnlp)

Challenge:	Effectively and efficiently handling complex realworld problems has become a key focus across industry and academia.
Approach:	They propose a tree-of-code framework that generates nodes through self-supervision and combines prompt and model exploration in a GT-free setting.
Outcome:	Experiments on two datasets with ten popular zero-shot LLMs show that Tree-of-Code boosts accuracy by nearly 20% over CodeAct with fewer than 1/4 turns.

Copied to clipboard

G Shahariar, Jia Chen, Jiachen Li, Yue Dong

Attributed and Predictive Entity Embedding for Fine-Grained Entity Typing in Knowledge Bases (C18-1)

Challenge:	Recent studies show that text-to-image models are vulnerable to adversarial perturbations .
Approach:	They investigate the impact of adversarial attacks on different POS tags within text prompts on T2I models.
Outcome:	The proposed model is vulnerable to adversarial perturbations with noun perturbations in text prompts.

Copied to clipboard

Hailong Jin, Lei Hou, Juanzi Li, Tiansi Dong

CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models (2025.findings-naacl)

Challenge:	Existing methods for identifying semantic type of entities are incomplete even in large knowledge bases.
Approach:	They propose an attributed and predictive entity embedding method which can fully utilize various kinds of information comprehensively.
Outcome:	Experiments on two real DBpedia datasets show that the proposed method outperforms 8 state-of-the-art methods with 4.0% improvement in Mi-F1 and 5.2% improvement in Ma-F1.

Copied to clipboard

Shangda Wu, Yashan Wang, Ruibin Yuan, Guo Zhancheng, Xu Tan, Ge Zhang, Monan Zhou, Jing Chen, Xuefeng Mu, Yuejie Gao, Yuanliang Dong, Jiafeng Liu, Xiaobing Li, Feng Yu, Maosong Sun

BFClass: A Backdoor-free Text Classification Framework (2021.findings-emnlp)

Challenge:	Current music information retrieval systems struggle to meet linguistic diversity challenges . current systems struggle with text queries in non-English languages .
Approach:	They propose a music information retrieval system that supports both ABC notation and MIDI . CLaMP 2 includes a multilingual text encoder and a multiple-modal music encoder .
Outcome:	The proposed system achieves state-of-the-art results in multilingual semantic search and music classification across modalities.

Copied to clipboard

Zichao Li, Dheeraj Mekala, Chengyu Dong, Jingbo Shang

Search-o1: Agentic Search-Enhanced Large Reasoning Models (2025.emnlp-main)

Challenge:	Various trigger design strategies have been explored to attack text classifiers, however, defending such attacks remains an open problem.
Approach:	They propose a backdoor-free training framework that poisons a subset of training data by injecting trigger patterns and setting their labels as the target labels.
Outcome:	The proposed framework can detect all the triggers, remove 95% of poisoned training samples with very limited false alarms, and achieve almost the same performance as the models trained on benign training data.

Copied to clipboard

Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, Zhicheng Dou

HEAL: An Empirical Study on Hallucinations in Embodied Agents Driven by Large Language Models (2025.findings-emnlp)

Challenge:	Large reasoning models (LRMs) have demonstrated impressive long stepwise reasoning capabilities through large-scale reinforcement learning.
Approach:	They propose a framework that enhances large reasoning models with an agentic retrieval-augmented generation mechanism and a Reason-in-Documents module for refining retrieved documents.
Outcome:	The proposed framework enhances LRMs with an agentic retrieval-augmented generation mechanism and Reason-in-Documents module for refining retrieved documents.

Copied to clipboard

Trishna Chakraborty, Udita Ghosh, Xiaopan Zhang, Fahim Faisal Niloy, Yue Dong, Jiachen Li, Amit Roy-Chowdhury, Chengyu Song

LitVISTA: A Benchmark for Narrative Orchestration in Literary Text (2026.acl-long)

Challenge:	Large language models are increasingly being adopted as the cognitive core of embodied agents.
Approach:	They propose a systematic study of hallucinations in large language models . they aim to understand to what extent hallucinos occur, what types trigger them .
Outcome:	The proposed model can induce hallucinations up to 40 higher than base prompts . the model fails to resolve scene-task inconsistencies, the study finds .

Copied to clipboard

Mingzhe Lu, Yiwen Wang, Yanbing Liu, Qi You, Chong Liu, Ruize Qin, Haoyu Dong, Wenyu Zhang, JiaRui Zhang, Yue Hu, Yunpeng Li

Challenge:	Existing large language models focus on causal coherence, neglecting the complex story arcs and orchestration inherent in human narratives.
Approach:	They propose a high-dimensional framework for narrative orchestration that unifies human and model perspectives while jointly characterizing narrative function and structure in a common space.
Outcome:	The proposed framework unifies human and model perspectives while jointly characterizing narrative function and structure in a common space.

ChatGLM-Math: Improving Math Problem-Solving in Large Language Models with a Self-Critique Pipeline (2024.findings-emnlp)

Copied to clipboard

Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Zhao Wenyi, Jie Tang, Yuxiao Dong

Numerical Reasoning

Challenge:	Large language models (LLMs) have shown excellent mastering of human language but struggle in real-world applications that require mathematical problem-solving.
Approach:	They propose a pipeline to train a general Math-Critique model from the LLM itself to provide feedback signals and employ rejective fine-tuning and direct preference optimization over the Llm's own generations for data collection.
Outcome:	The proposed pipeline outperforms existing LLMs that could be two times larger.

Memory-Efficient Differentiable Transformer Architecture Search (2021.findings-acl)

Copied to clipboard

Yuekai Zhao, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen

Data-to-text Generation with Entity Modeling (P19-1)

Challenge:	Current neural architecture search methods suffer from huge computational cost.
Approach:	They propose a reversible recursive backpropagation algorithm that uses the last layer to store the outputs of the network.
Outcome:	The proposed algorithm outperforms standard Transformers on three sequence-to-sequence datasets.

Copied to clipboard

Ratish Puduppully, Li Dong, Mirella Lapata

PhysPRM: A Generative Process Reward Model with Fine-grained Diagnosis for Physics Problem Solving (2026.findings-acl)

Challenge:	Recent approaches to data-to-text generation have shown great promise thanks to the use of large-scale datasets and the application of neural network architectures which are trained end-to end.
Approach:	They propose an entity-centric neural architecture for data-to-text generation which uses hierarchical attention to create entity-specific representations which are dynamically updated.
Outcome:	The proposed model outperforms baselines in automatic and human evaluation on the RotoWire benchmark and a five-times larger dataset on the baseball domain.

Copied to clipboard

Yuxuan Dong, Xinyu Zhang, Lingling Zhang, Han Lai, Pengyu Li, Bifan Wei, Yaqiang Wu, Jun Liu

U-CORE: A Unified Deep Cluster-wise Contrastive Framework for Open Relation Extraction (2023.tacl-1)

Challenge:	Existing Large Language Models (LLMs) struggle with physics problem solving due to difficulties in decoding implicit constraints and maintaining physical consistency.
Approach:	They propose a Generative PRM that treats evaluation as a generative task . it produces fine-grained diagnoses comprising critiques, final judgments, and specific error types .
Outcome:	The proposed model improves performance across seven benchmarks in Best-of-N and critique refinement strategies.

Copied to clipboard

Jie Zhou, Shenpo Dong, Yunxin Huang, Meihan Wu, Haili Li, Jingnan Wang, Hongkui Tu, Xiaodong Wang

TDEER: An Efficient Translating Decoding Schema for Joint Extraction of Entities and Relations (2021.emnlp-main)

Challenge:	Existing methods for Relation Extraction (RE) are limited due to the overlap between predefined and undefined relations.
Approach:	They propose a unified framework for both Zero-shot and Unsupervised Relation Extraction tasks by leveraging techniques from Contrastive Learning and Clustering.
Outcome:	The proposed framework improves on three well-known datasets showing an average improvement of 7.35% ARI on Zero-shot ORE tasks and 15.24% ARI for Unsupervised ORE.

Copied to clipboard

Xianming Li, Xiaotian Luo, Chenghao Dong, Daichuan Yang, Beidi Luan, Zhen He

GroundingGPT: Language Enhanced Multi-modal Grounding Model (2024.acl-long)

Challenge:	Existing methods to extract entities and relations from unstructured texts are difficult to handle due to the overlapping triple problem.
Approach:	They propose a translation decoding schema for joint extraction of entities and relations from unstructured texts to form factual triples.
Outcome:	The proposed model can handle the overlapping triple problem, and is 2 times faster than the state-of-the-art models.

Copied to clipboard

Zhaowei Li, Qi Xu, Dong Zhang, Hang Song, YiQing Cai, Qi Qi, Ran Zhou, Junting Pan, Zefeng Li, Vu Tu, Zhida Huang, Tao Wang

Your Reasoning Model is Secretly a Reward Model - Optimization-Free Verification from Experience (2026.acl-long)

Challenge:	Existing multi-modal large language models focus on capturing global information while neglecting the fine-grained local information in multimodal inputs.
Approach:	They propose an end-to-end language enhanced multi-modal grounding model that performs fine-grained grounding tasks for image, video and audio.
Outcome:	The proposed model achieves impressive fine-grained understanding of multi-modal inputs while maintaining or improving its global comprehension capabilities.

Copied to clipboard

Zhenwen Liang, Ruosen Li, Yujun Zhou, Linfeng Song, Dian Yu, Xinya Du, Haitao Mi, Dong Yu

DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimization (2025.findings-naacl)

Challenge:	Existing verifiers operate on the surface text or on confidence proxies derived from token probabilities, which can be brittle.
Approach:	They propose a training-free, non-parametric verifier that summarizes each reasoning trace by an activation delta and compares it to two class centroids computed from labeled experience.
Outcome:	The proposed model improves selection and reranking on large and less-calibrated models.

Copied to clipboard

Pucheng Dang, Xing Hu, Dong Li, Rui Zhang, Qi Guo, Kaidi Xu

LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation (2025.acl-long)

Challenge:	Existing text-to-image (T2I) synthesis diffusion models raise misuse concerns, particularly in creating prohibited or not-safe-for-work (NSFW) images.
Approach:	They propose a method which uses zeroth order optimization to procure gradient approximations and harnesses both C-PRV and D-PRv to enhance attack prompts within a discrete prompt space.
Outcome:	The proposed method achieves an 8.5% higher average attack success rate than previous works on multiple state-of-the-art safety mechanisms.

Copied to clipboard

Zican Dong, Junyi Li, Jinhao Jiang, Mingyu Xu, Xin Zhao, Bingning Wang, Weipeng Chen

HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression (2021.emnlp-main)

Challenge:	Large language models (LLMs) have extended context windows through scaling positional encodings and lightweight continual pre-training, but performance degradation is still not fully explored.
Approach:	They propose a novel approach to reduce short-text performance degradation by minimizing distribution drift in hidden states and attention scores.
Outcome:	The proposed approach minimizes the distribution discrepancy between the extended and original models while maintaining or even enhancing the model's long-context abilities.

Copied to clipboard

Chenhe Dong, Yaliang Li, Ying Shen, Minghui Qiu

Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs (2026.acl-long)

Challenge:	Large pre-trained language models (PLMs) have shown overwhelming performances on many tasks, but their large size and slow inference speed have hindered practical deployments.
Approach:	They propose a hierarchical relational knowledge distillation method to capture hierarchic and domain relational information.
Outcome:	The proposed method outperforms existing methods on multi-domain datasets and is highly reproducible.

Copied to clipboard

Tingchao Fu, Wenkai Wang, Fanxiao Li, Huadong Zhang, Jinhong Zhang, Dayang Li, Yunyun Dong, Renyang Liu, Wei Zhou

Learning from Diverse Reasoning Paths with Routing and Collaboration (2025.emnlp-main)

Challenge:	Existing knowledge editing paradigms suffer from editing decoupling failures . entity knowledge is sequestered into disentangled modality-specific pathways .
Approach:	They propose a method that explicitly disentangles and localizes modality-specific neuron groups for targeted knowledge.
Outcome:	The proposed method outperforms baselines in reliability and consistency while preserving model locality.

Copied to clipboard

Zhenyu Lei, Zhen Tan, Song Wang, Yaochen Zhu, Zihan Chen, Yushun Dong, Jundong Li

Multi-Document Summarization with Determinantal Point Processes and Contextualized Representations (D19-54)

Challenge:	Recent studies suggest that the reasoning abilities of large language models (LLMs) grows with model size and pre-training data.
Approach:	They propose to combine quality filtering, conditional routing, and cooperative peer teaching to transfer knowledge from powerful teacher models to compact and transparent students.
Outcome:	Experiments show that QR-Distill is superior to traditional methods.

Copied to clipboard

Sangwoo Cho, Chen Li, Dong Yu, Hassan Foroosh, Fei Liu

Summarization

Challenge:	Determinantal point processes (DPP) is one of the best performing techniques for extractive summarization.
Approach:	They propose to combine determinantal point processes with surface indicators for effective identification of summary-worthy sentences.
Outcome:	The determinantal point processes (DPP) framework is one of the best performing in summarization competitions.

EduBench: A Comprehensive Benchmarking Dataset for Evaluating Large Language Models in Diverse Educational Scenarios (2026.acl-long)

Copied to clipboard

Bin Xu, Yu Bai, Huashan Sun, Yiguan Lin, Siming Liu, Xinyue Liang, Yaolin Li, Zhuangzhi Dong, Jingren Zhang, Yufan Deng, Xinyu Zou, Yang Gao, Heyan Huang

Self-SoftCoT: A Self-Consistent Framework via Position-Aware Latent Space Reinforcement Learning (2026.acl-long)

Challenge:	Existing benchmarks that focus on knowledge-intensive tasks do not reflect diverse educational scenarios.
Approach:	They propose a benchmark that incorporates 9 major scenarios and 4,000 educational contexts.
Outcome:	The proposed model performs comparable to state-of-the-art large models on the test set.

Copied to clipboard

Liangliang Dong, Lianlei Shan, Shuaimin Li

Introducing Semantics into Speech Encoders (2023.acl-long)

Challenge:	Existing Continuous reasoning approaches rely on external auxiliary models, resulting in complex deployment and fractured inference pipelines.
Approach:	They propose a self-contained framework that enables a frozen LLM to internally generate and consume latent thoughts without external assistants.
Outcome:	The proposed framework outperforms SoftCoT models on five reasoning benchmarks.

Copied to clipboard

Derek Xu, Shuyan Dong, Changhan Wang, Suyoun Kim, Zhaojiang Lin, Bing Liu, Akshat Shrivastava, Shang-Wen Li, Liang-Hsuan Tseng, Guan-Ting Lin, Alexei Baevski, Hung-yi Lee, Yizhou Sun, Wei Wang

Coarse-to-Fine Decoding for Neural Semantic Parsing (P18-1)

Challenge:	Existing self-supervised speech encoders contain primarily acoustic rather than semantic information.
Approach:	They propose a task-agnostic unsupervised way to incorporate semantic information from large language model (LLM) systems into self-supervised speech encoders without labeled audio transcriptions.
Outcome:	The proposed approach improves spoken language understanding (SLU) performance by over 5% on intent classification (IC), with modest gains in named entity resolution (NER) and slot filling (SF), and spoken question answering (SQA) score by over 22%.

Copied to clipboard

Li Dong, Mirella Lapata

KELE: A Multi-Agent Framework for Structured Socratic Teaching with Large Language Models (2025.findings-emnlp)

Challenge:	Experimental results show that semantic parsing is more efficient than using simple decoders.
Approach:	They propose a structure-aware neural architecture which decomposes the semantic parsing process into two stages.
Outcome:	The proposed architecture consistently improves performance on four datasets characteristic of different domains and meaning representations.

Copied to clipboard

Xian Peng, Pan Yuan, Dong Li, Junlong Cheng, Qin Fang, Zhi Liu

MorphoBench: A Benchmark with Difficulty Adaptive to Model Reasoning (2026.findings-acl)

Challenge:	Socratic teaching places high demands on teachers’ expertise and real-time feedback capabilities, making it difficult to scale in large educational settings.
Approach:	They propose a multi-agent framework for structured Socratic teaching with LLMs that integrates a structured SocRule and a consultant-teacher collaborative teaching mechanism.
Outcome:	The proposed framework outperforms existing LLMs in natural language generation and dialogue comprehension in the classroom.

Copied to clipboard

Xukai Wang, Xuanbo Liu, Mingrui Chen, Haitian Zhong, Xuanlin Yang, Bohan Zeng, Jinbo Hu, Hao Liang, Junbo Niu, Xuchen Li, Ruitao Wu, Ruichuan An, Yang Shi, Liu Liu, Qiang Liu, Zhouchen Lin, Xu-Yao Zhang, Wentao Zhang, Bin Dong

READoc: A Unified Benchmark for Realistic Document Structured Extraction (2025.findings-acl)

Challenge:	Existing benchmarks designed to evaluate the reasoning capabilities of large models are limited in scope and lack flexibility to adapt difficulty according to evolving reasoning capacities of models.
Approach:	They propose a benchmark that incorporates multidisciplinary questions to evaluate the reasoning capabilities of large models and can adjust and update question difficulty based on the reasoning abilities of advanced models.
Outcome:	The proposed benchmark incorporates multidisciplinary questions to evaluate the reasoning capabilities of large models and can adjust and update question difficulty based on the reasoning abilities of advanced models.

Copied to clipboard

Zichao Li, Aizier Abulaiti, Yaojie Lu, Xuanang Chen, Jia Zheng, Hongyu Lin, Xianpei Han, Shanshan Jiang, Bin Dong, Le Sun

Mitigating Over-Refusal in Aligned Large Language Models via Inference-Time Activation Energy (2026.acl-long)

Challenge:	Document Structured Extraction (DSE) is a field of document structure analysis that aims to extract structured content from raw documents.
Approach:	They propose a benchmark to evaluate document structured extraction systems by converting unstructured PDFs into semantically rich Markdown.
Outcome:	The proposed benchmark is based on 3,576 diverse and real-world documents from arXiv, GitHub, and Zenodo.

Copied to clipboard

Eric Hanchen Jiang, Weixuan Ou, Run Liu, Shengyuan Pang, Guancheng Wan, Ranjie Duan, Wei Dong, Kai-Wei Chang, XiaoFeng Wang, Ying Nian Wu, Xinfeng Li

APPL: A Prompt Programming Language for Harmonious Integration of Programs and Large Language Model Prompts (2025.acl-long)

Challenge:	Existing safety alignment techniques prioritize mitigating harmful responses at the expense of overcautious behavior, leading models to incorrectly refuse benign requests.
Approach:	They propose a fine-tuning free framework to improve safety and reduce false refusals by dynamic, inference-time intervention.
Outcome:	The proposed framework raises compliance on the ORB-H benchmark from 57.3% to 82.6% while maintaining the baseline safety performance.

Copied to clipboard

Honghua Dong, Qidong Su, Yubo Gao, Zhaoyu Li, Yangjun Ruan, Gennady Pekhimenko, Chris J. Maddison, Xujie Si

ProcWorld: Benchmarking Large Model Planning in Reachability-Constrained Environments (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) are capable of handling diverse tasks with well-crafted prompts and integration of external tools.
Approach:	They propose a prompt programming language that acts as a bridge between computer programs and LLMs by providing convenient conventions between them.
Outcome:	The proposed language is intuitive, concise, and efficient through representative scenarios including Chain-of-Thought with self-consistency (CoT-SC) and ReAct tool-use agent.

Copied to clipboard

Dong Wang, Xinghang Li, Zhengshen Zhang, Jirong Liu, Xiao Ma, Hanbo Zhang, Tao Kong, Huaping Liu

BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models (2024.lrec-main)

Challenge:	Existing benchmarks for embodied spatial reasoning and long-term planning are non-trivial due to the combinatorial complexity of long-horizon abstract reasoning.
Approach:	They propose a large-scale benchmark for partially observable embodied spatial reasoning and long-term planning with large language models and vision language models.
Outcome:	The proposed model performs better in 16 task types, 5,000 rooms, and over 10 million evaluation trajectories with diverse data distribution.

Copied to clipboard

Zican Dong, Tianyi Tang, Junyi Li, Wayne Xin Zhao, Ji-Rong Wen

Automated Generation of Accurate & Fluent Medical X-ray Reports (2021.emnlp-main)

Challenge:	Existing long context models suffer from performance decline when the input text exceeds their length limit.
Approach:	They propose a multi-task long context benchmark to evaluate LLMs' long context ability using 10 datasets from 5 different NLP tasks.
Outcome:	The proposed model covers 5 domains and core capacities of large language models.

Copied to clipboard

Hoang Nguyen, Dong Nie, Taivanbat Badamdorj, Yujie Liu, Yingying Zhu, Jason Truong, Li Cheng

A Comprehensive Graph Framework for Question Answering with Mode-Seeking Preference Alignment (2025.findings-acl)

Challenge:	Existing medical report generation efforts focus on producing human-readable reports, yet the generated text may not be well aligned to the clinical facts.
Approach:	They propose to automate the generation of medical reports from chest X-ray image inputs . medical reports are the primary medium, which physicians communicate findings from scans - authors say .
Outcome:	The proposed method achieves fluency and clinical accuracy on common metrics.

Copied to clipboard

Quanwei Tang, Sophia Yat Mei Lee, Junshuang Wu, Dong Zhang, Shoushan Li, Erik Cambria, Guodong Zhou

Structured Data in NLP

Challenge:	Existing studies struggle with achieving global understanding of large language models . GraphMPA is a graph-based framework with mode-seeking preference alignment .
Approach:	They propose a graph-based framework with mode-seeking preference alignment to improve model outputs.
Outcome:	The proposed framework constructs a hierarchical document graph mimicking human cognitive processes for information understanding and synthesis.

Controllable Natural Language Generation with Contrastive Prefixes (2022.findings-acl)

Copied to clipboard

Jing Qian, Li Dong, Yelong Shen, Furu Wei, Weizhu Chen

LLM-Powered Test Case Generation for Detecting Bugs in Plausible Programs (2025.acl-long)

Challenge:	Existing work on controllable natural language generation has focused on fine-tuning existing models or using attribute discriminators.
Approach:	They propose a lightweight framework for controllable GPT2 generation that utilizes attribute-specific vectors to steer natural language generation.
Outcome:	The proposed framework can guide generation towards desired attributes while keeping high linguistic quality.

Copied to clipboard

Kaibo Liu, Zhenpeng Chen, Yiyang Liu, Jie M. Zhang, Mark Harman, Yudong Han, Yun Ma, Yihong Dong, Ge Li, Gang Huang

Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues (2022.acl-long)

Challenge:	TrickCatcher generates test cases that pass existing tests yet contain bugs . a recent study found that tricky bugs are not detected by test suites .
Approach:	They propose an LLM-powered approach to generating test cases for uncovering bugs in plausible programs . they use a PUT and specification to generate program variants, an input generator and an Llm to construct test inputs .
Outcome:	The proposed approach achieves recall, precision, and F1 scores that are 1.80, 2.65, and 1.66 . trickCatcher generates program variants based on the program under test and its specification .

Copied to clipboard

Qingxiu Dong, Ziwei Qin, Heming Xia, Tian Feng, Shoujie Tong, Haoran Meng, Lin Xu, Zhongyu Wei, Weidong Zhan, Baobao Chang, Sujian Li, Tianyu Liu, Zhifang Sui

CalibraEval: Calibrating Prediction Distribution to Mitigate Selection Bias in LLMs-as-Judges (2025.acl-long)

Challenge:	Existing work in vision language cross-modal reasoning uses binary or multi-choice classification based on source image and textual query.
Approach:	They propose a task where a textual premise is the background presumption on each source image.
Outcome:	The proposed task is based on a dataset of 15,360 movie screenshots and human-curated premise templates from 6 pre-defined categories.

Copied to clipboard

Haitao Li, Junjie Chen, Qingyao Ai, Zhumin Chu, Yujia Zhou, Qian Dong, Yiqun Liu

DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models (2023.emnlp-main)

Challenge:	Empirical evaluations of large language models demonstrate that they improve performance in a wide range of tasks.
Approach:	They propose a label-free method for mitigating selection bias during inference by reformulating debiasing as an optimization task.
Outcome:	The proposed method mitigates selection bias and improves performance compared to existing methods.

Copied to clipboard

Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, Deyi Xiong

Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLMs (2025.findings-emnlp)

Challenge:	Existing studies have demonstrated that pretrained language models memorize and regurgitate a significant portion of training data, including atypical data points that appear only once in the training data.
Approach:	They propose a method to locate and erase risky neurons in order to eliminate the impact of privacy data in the model in batches.
Outcome:	The proposed method eliminates the impact of privacy data in the model in batches without affecting the model's performance.

Copied to clipboard

Chenkun Tan, Pengyu Wang, Shaojun Zhou, Botian Jiang, Zhaowei Li, Dong Zhang, Xinghao Wang, Yaqian Zhou, Xipeng Qiu

MMDocIR: Benchmarking Multimodal Retrieval for Long Documents (2025.emnlp-main)

Challenge:	Recent advances in multimodal large language models focus on improving performance . however, language prior conflict leads to suboptimal vision-language alignment .
Approach:	They propose a method to decouple the alignment process from language prior interference . they use a proxy LLM to detach from language interference during pretraining .
Outcome:	The proposed method improves training performance and generalizes training data.

Copied to clipboard

Kuicai Dong, Yujing Chang, Derrick Goh Xin Deik, Dexun Li, Ruiming Tang, Yong Liu

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption (2022.findings-acl)

Challenge:	Existing benchmarks for multimodal document retrieval are lacking for evaluating performance of systems.
Approach:	They propose a benchmark that evaluates page-level and layout-level retrieval tasks . they use a rich dataset featuring 1,685 questions annotated by experts .
Outcome:	The proposed benchmark outperforms existing benchmarks in page-level and layout-level retrieval tasks.

Copied to clipboard

Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, Furu Wei

Harnessing Large Language Models for Disaster Management: A Survey (2025.findings-acl)

Challenge:	enabling pre-trained models inference on ciphertext data is difficult due to the complex computations in transformer blocks.
Approach:	They propose an approximation approach for transformers which enables inference on ciphertext data.
Outcome:	The proposed approach can infer pre-trained models on encrypted data with negligible performance drop but enjoy theory-guaranteed privacy-preserving advantage.

Copied to clipboard

Zhenyu Lei, Yushun Dong, Weiyu Li, Rong Ding, Qi R. Wang, Jundong Li

AGTAO: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains, including their emerging role in mitigating threats to human life, infrastructure, and the environment during natural disasters.
Approach:	They propose a taxonomy that categorizes existing LLMs based on disaster phases and application scenarios to provide valuable insights for the research community and practitioners .
Outcome:	The proposed taxonomy categorizes existing LLMs based on disaster phases and application scenarios.

Copied to clipboard

Pengyu Li, Lingling Zhang, Zhitao Gao, Yanrui Wu, Yuxuan Dong, Huan Liu, Bifan Wei, Jun Liu

Zero-shot Cross-lingual NER via Mitigating Language Difference: An Entity-aligned Translation Perspective (2025.findings-emnlp)

Challenge:	Large Language Models (LLMs) unintentionally memorize sensitive data, posing privacy and security risks.
Approach:	They propose a framework that reconciles unlearning efficacy and utility preservation by using a latent-space gating mechanism to simulate internal recovery attempts.
Outcome:	The proposed framework achieves superior trade-off between unlearning efficacy and model utility.

Copied to clipboard

Zhihao Zhang, Sophia Yat Mei Lee, Dong Zhang, Shoushan Li, Guodong Zhou

MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning (2024.acl-long)

Challenge:	Existing approaches to cross-lingual Named Entity Recognition focus on Latin script language (LSL) for non-Latin script language, performance often degrades due to deep structural differences.
Approach:	They propose an entity-aligned translation approach to align entities between NSL and English .
Outcome:	The proposed approach aims to transfer knowledge from high-resource languages to low-resourced languages.

Copied to clipboard

Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, Chang Zhou

Curse of Knowledge: Your Guidance and Provided Knowledge are biasing LLM Judges in Complex Evaluation (2025.findings-emnlp)

Challenge:	In math reasoning with large language models, fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective.
Approach:	They propose to fine-tune data augmentation by query evolution and diverse reasoning paths.
Outcome:	The proposed model achieves new state-of-the-art on GSM8K and MATH.

Copied to clipboard

Weiyuan Li, Xintao Wang, Siyu Yuan, Rui Xu, Jiangjie Chen, Qingqing Dong, Yanghua Xiao, Deqing Yang

Challenge:	a recent study has focused on simple settings, but their reliability in complex tasks remains understudied.
Approach:	They propose to use large language models as judges to evaluate reliability in complex tasks . they use a challenge benchmark to expose and quantify Auxiliary Information Induced Biases .
Outcome:	The proposed benchmark exposes and quantifies Auxiliary Information Induced Biases across 12 basic and 3 advanced scenarios.

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023.findings-acl)

Copied to clipboard

Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei

RAG-Critic: Leveraging Automated Critic-Guided Agentic Workflow for Retrieval Augmented Generation (2025.acl-long)

Challenge:	Large pretrained language models have shown surprising in-context learning ability . despite the great success in performance, its working mechanism remains unclear .
Approach:	They explain language models as meta-optimizers and understand in-context learning as implicit finetuning . they find that Transformer attention has a dual form of gradient descent .
Outcome:	The proposed model can predict labels for unseen inputs without parameter updates . the proposed model outperforms smaller models with a single parameter update .

Copied to clipboard

Guanting Dong, Jiajie Jin, Xiaoxi Li, Yutao Zhu, Zhicheng Dou, Ji-Rong Wen

Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains (2026.acl-long)

Challenge:	Recent advances in large language models (LLMs) have demonstrated remarkable performance across a wide range of downstream tasks.
Approach:	They propose a framework that leverages a critic-guided agentic workflow to improve RAG capabilities autonomously.
Outcome:	The proposed framework improves RAG capabilities autonomously by leveraging a critic-guided agentic workflow.

Copied to clipboard

Zhonghang Yuan, Zhefan Wang, Fang Hu, Zihong Chen, Jinzhe Li, Gang Li, Jie Ying, Huanjun Kong, Songyang Zhang, Nanqing Dong

Entity-level Interaction via Heterogeneous Graph for Multimodal Named Entity Recognition (2022.findings-emnlp)

Challenge:	Recent large language models (LLMs) have demonstrated remarkable progress in reasoning, but their applications on knowledge-intensive domains have not been explored due to the scarcity of high-quality verifiable data.
Approach:	They propose a framework that extends reinforcement learning with verifiable rewards (RLVR) to knowledge-intensive domains through automated verififiability data synthesis while enabling verification of the LLM's reasoning process.
Outcome:	Extensive experiments show that the proposed framework enhances the reasoning of large language models in knowledge-intensive domains without significantly compromising the model’s general capabilities.

Copied to clipboard

Gang Zhao, Guanting Dong, Yidong Shi, Haolong Yan, Weiran Xu, Si Li

Interleaved Latent Visual Reasoning with Selective Perceptual Modeling (2026.acl-long)

Challenge:	Existing methods for name-based entity recognition neglect the integrity of entity semantics and conduct cross-modal interaction at token-level.
Approach:	They propose a multimodal named entity recognition model that captures visual information and fuses it into tokens to rid non-entity tokens of visual noise.
Outcome:	The proposed model captures entity-related visual information and fuses it into tokens . it eliminates visual noise and makes non-entity tokens easily misidentified as entities .

Copied to clipboard

Shuai Dong, Siyuan Wang, Xingyu Liu, Chenglin Li, Haowen Hou, Zhongyu Wei

Multi-Scale Spectral Selection and Entropy-Guided Uncertainty Fusion for Multimodal Rumor Detection (2026.findings-acl)

Challenge:	Existing approaches to interleaved reasoning are limited by the cost of re-encoding pixel-dense images.
Approach:	They propose a framework that unifies dynamic state evolution with precise perceptual modeling.
Outcome:	The proposed framework outperforms existing approaches on multimodal reasoning benchmarks.

Copied to clipboard

Zongliang Han, Wenyu Guo, Guoqing Jin, Yang Liu, Fan Li, Dong Yu, Yan Song, null Zhangfengzhen

HIE-SQL: History Information Enhanced Network for Context-Dependent Text-to-SQL Semantic Parsing (2022.findings-acl)

Challenge:	Existing methods for multimodal content detection fail to capture cross-modal semantic inconsistencies and ignore inherent noise in multimodal features.
Approach:	They propose a multimodal rumor detection method based on a frequency domain spectral selection method and entropy-guided uncertainty fusion method to capture cross-modal semantic inconsistencies.
Outcome:	The proposed method outperforms state-of-the-art methods in multimodal rumor detection . it shows stronger detection capability and robustness on multiple datasets .

Copied to clipboard

Yanzhao Zheng, Haibin Wang, Baohua Dong, Xingjun Wang, Changshan Li

Focused-DPO: Enhancing Code Generation Through Focused Preference Optimization on Error-Prone Points (2025.findings-acl)

Challenge:	Recent studies focus on context-dependent text-to-SQL task but fail to exploit both . et al., 2019; xu e. al.; yu y., 2021) focus on the context-independent text to SQL task .
Approach:	They propose a history information enhanced text-to-SQL model to exploit context dependence information from history utterances and the last predicted SQL query.
Outcome:	The proposed model improves performance on two context-dependent text-to-SQL benchmarks.

Copied to clipboard

Kechi Zhang, Ge Li, Jia Li, Yihong Dong, Jia Li, Zhi Jin

Saber: Efficient Sampling with Adaptive Acceleration and Backtracking Enhanced Remasking for Diffusion Language Model in Code Generation (2026.acl-long)

Challenge:	Current code generation models produce errors concentrated at specific error-prone points, affecting accuracy of code.
Approach:	They propose a framework that focuses preference optimization on error-prone areas . focused-DPO improves the accuracy and reliability of code generation by reducing common errors .
Outcome:	The proposed framework improves code generation by focusing on error-prone areas.

Copied to clipboard

Yihong Dong, Zhaoyu Ma, Xue Jiang, Zhiyuan Fan, Jiaru Qian, Yongmin Li, Jianha Xiao, Zhi Jin, Ge Li

EvoCoT: Overcoming the Exploration Bottleneck in Reinforcement Learning for LLMs (2026.findings-acl)

Challenge:	Diffusion language models (DLMs) offer advantages in parallel generation and bidirectional context modeling, but they face a critical trade-off between inference speed and output quality for tasks with strict structural constraints such as code generation.
Approach:	They propose an efficient sampling algorithm that reduces the number of tokens unmasked per step based on the model’s evolving confidence.
Outcome:	The proposed method improves Pass@1 accuracy by 1.9% while achieving 251.4% inference speedup.

Copied to clipboard

Huanyu Liu, Jia Li, Yihong Dong, Chang Yu, Taozhi Chen, Lecheng Wang, Yongding Tao, Bin Gu, Ge Li

Persona-E²: A Human-Grounded Dataset for Personality-Shaped Emotional Responses to Textual Events (2026.acl-long)

Challenge:	Existing approaches to reinforcement learning with verifiable reward (RLVR) are limited by difficulty or lack of exploration.
Approach:	They propose a self-evolving curriculum learning framework based on chain-of-thought reasoning optimization that constrains exploration space by self-generating and verifying CoT trajectories.
Outcome:	The proposed framework enables LLMs to solve previously unsolved problems without external supervision and is compatible with various RL fine-tuning methods.

Copied to clipboard

Yuqin Yang, Haowu Zhou, Haoran Tu, Zhiwen Hui, Shiqi Yan, HaoYang Li, Dong She, Xianrong Yao, Yang Gao, Zhanpeng Jin

Challenge:	A critical bottleneck is the lack of ground-truth human data to link personality traits to emotional shifts.
Approach:	They propose a large-scale dataset to capture reader-based emotional variations across news, social media, and life narratives.
Outcome:	The proposed model captures reader-based emotional variations across news, social media, and life narratives.

StableMoE: Stable Routing Strategy for Mixture of Experts (2022.acl-long)

Copied to clipboard

Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei

Challenge:	Existing learning-to-route methods suffer from the routing fluctuation issue . with the model scale growing, training speed will go slower and memory requirements are heavy .
Approach:	They propose a Mixture-of-Experts technique that can scale up the model size of Transformers with an affordable computational overhead.
Outcome:	The proposed method outperforms existing learning-to-route methods on language modeling and multilingual machine translation.

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition (2024.acl-long)

Copied to clipboard

Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, Jingren Zhou

ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdom (2025.emnlp-main)

Challenge:	supervised fine-tuning (SFT) is a technique used to enhance multiple abilities in large language models.
Approach:	They propose to study the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during supervised fine-tuning.
Outcome:	The proposed model improves math reasoning and code generation with increasing data amount . the proposed model size and SFT strategies can be used to learn multiple skills with different scaling patterns.

Copied to clipboard

Jingqi Zhou, Sheng Wang, Jingwei Dong, Kai Liu, Lei Li, Jiahui Gao, Jiyue Jiang, Lingpeng Kong, Chuan Wu

Investigating Learning Dynamics of BERT Fine-Tuning (2020.aacl-main)

Challenge:	Large vision-language models often prioritize language knowledge over image information on visual reasoning tasks, incurring performance degradation.
Approach:	They propose a visual reasoning framework that decouples vision-reasoning capabilities and multi-run proactive perception.
Outcome:	The proposed framework outperforms existing models on benchmarks for open-source and closed-source models with 13.2% performance gain.

Copied to clipboard

Yaru Hao, Li Dong, Furu Wei, Ke Xu

SpeechGPT: Empowering Large Language Models with Intrinsic Cross-Modal Conversational Abilities (2023.findings-emnlp)

Challenge:	Recent studies have shown that the fine-tuning process improves performance on downstream tasks.
Approach:	They propose two new pre-training tasks to improve the model performance on downstream tasks.
Outcome:	The proposed model achieves state-of-the-art on a wide array of NLP tasks.

Copied to clipboard

Dong Zhang, Shimin Li, Xin Zhang, Jun Zhan, Pengyu Wang, Yaqian Zhou, Xipeng Qiu

Rethinking Repetition Problems of LLMs in Code Generation (2025.acl-long)

Challenge:	Existing multi-modal large language models typically adopt the cascade paradigm, preventing inter-modal knowledge transfer.
Approach:	They propose a large language model with intrinsic cross-modal conversational abilities . they construct a cross-text speech instruction dataset and employ a three-stage training strategy .
Outcome:	The proposed model can follow cross-modal human instructions and handle multiple modalities with one model.

Copied to clipboard

Yihong Dong, Yuchen Liu, Xue Jiang, Bin Gu, Zhi Jin, Ge Li

Calibrating Factual Knowledge in Pretrained Language Models (2022.findings-emnlp)

Challenge:	Recent studies have focused on content repetition, but structural repetition is a more prevalent problem in code generation.
Approach:	They propose a decoding approach that eliminates repetition problems in code generation by identifying grammar rules and strategically decaying the likelihood of critical tokens that contribute to repetitions.
Outcome:	The proposed approach outperforms baselines and humanEval benchmarks on CodeRepetEval dataset and MBPP benchmarks, effectively reducing repetitions and enhancing the quality of generated code.

Copied to clipboard

Qingxiu Dong, Damai Dai, Yifan Song, Jingjing Xu, Zhifang Sui, Lei Li

Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis (2024.findings-naacl)

Challenge:	Existing studies show that Pretrained Language Models can store factual knowledge, but facts stored in PLMs are not always correct.
Approach:	They propose a lightweight method to calibrate factual knowledge in PLMs without re-training from scratch.
Outcome:	The proposed method can be used to calibrate factual knowledge in PLMs without re-training from scratch.

Copied to clipboard

Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li

DataSciBench: An LLM Agent Benchmark for Data Science (2026.findings-acl)

Challenge:	Existing studies show that large language models (LLMs) can handle multilingual machine translation (MMT) However, the multilingual translation ability of LLMs remains under-explored.
Approach:	They evaluate eight popular LLMs including ChatGPT and GPT-4 to determine their performance in multilingual machine translation.
Outcome:	The proposed model can generate moderate translation even on zero-resource languages and cross-lingual exemplars can provide better task guidance for low-resourced translation than exemplar in the same language pairs.

Copied to clipboard

Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, Yisong Yue

Challenge:	Existing benchmarks focus on single task, simple evaluation metrics, and readily available ground truth (GT) DataSciBench is built on curated, natural, and challenging prompts with complex evaluation criteria and uncertain GT.
Approach:	They propose a benchmark for evaluating Large Language Models in data science that integrates LLM-based self-consistency and human verification to ensure accuracy.
Outcome:	The proposed framework outperforms open-source models in all metrics and offers rigorous insights into LLM strengths and weaknesses.

Pre-Training to Learn in Context (2023.acl-long)

Copied to clipboard

Yuxian Gu, Li Dong, Furu Wei, Minlie Huang

A Survey on In-context Learning (2024.emnlp-main)

Challenge:	Pre-trained language models are not explicitly trained to learn in context.
Approach:	They propose a framework to enhance in-context learning by pre-training language models on a large collection of "intrinsic tasks" they evaluate the in-constitution learning performance of the model trained with PICL on seven widely-used text classification datasets and the Super-NaturalInstrctions benchmark .
Outcome:	The proposed framework outperforms larger language models with nearly 4x parameters on seven widely-used datasets and the Super-NaturalInstrctions benchmark.

Copied to clipboard

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui

EditNTS: An Neural Programmer-Interpreter Model for Sentence Simplification through Explicit Editing (P19-1)

Challenge:	In-context learning (ICL) is a new paradigm for natural language processing . large language models (LLMs) demonstrate the ability to learn from a few examples .
Approach:	They propose to explore ICL to evaluate and extrapolate the ability of large language models.
Outcome:	The proposed methods can be used to evaluate and extrapolate the ability of large language models.

Copied to clipboard

Yue Dong, Zichao Li, Mehdi Rezagholizadeh, Jackie Chi Kit Cheung

Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats (2026.acl-long)

Challenge:	Current sentence simplification systems are variants of sequence-to-sequence models adopted from machine translation.
Approach:	They propose a sentence simplification model that learns explicit edit operations via a neural programmer-interpreter approach.
Outcome:	The proposed model outperforms state-of-the-art models on three benchmark text simplification corpora in terms of SARI (+0.95 WikiLarge, +1.89 WikiSmall, -1.41 Newsela)

Copied to clipboard

Manyi Zhang, Ji-Fu Li, Zhongao Sun, Haoli Bai, Hui-Ling Zhen, Zhenhua Dong, Xianzhi Yu

Confidence Modeling for Neural Semantic Parsing (P18-1)

Challenge:	Microscaling Floating-Point (MXFP) is a low-precision format for large language models (LLMs).
Approach:	They conduct systematic evaluations of PTQ under Microscaling Floating-Point (MXFP) . they find that MXFP8 consistently achieves near-lossless performance .
Outcome:	The proposed method achieves near-lossless performance while MXFP4 introduces substantial accuracy degradation and remains challenging.

Copied to clipboard

Li Dong, Chris Quirk, Mirella Lapata

AutoAlign: Get Your LLM Aligned with Minimal Annotations (2025.acl-demo)

Challenge:	Experimental results show that neural semantic parsers are difficult to interpret due to their complexity.
Approach:	They propose to use confidence models to estimate predictions for neural semantic parsers . they outline three major causes of uncertainty and use metrics to quantify them .
Outcome:	The proposed model outperforms a widely used method that relies on posterior probability and improves interpretation quality.

Copied to clipboard

Xinyu Lu, Dong Xu, Chunkang Zhang, Xinyan Guan, Junxiang Wang, Qingyu Zhang, Pengbo Wang, Yingzhi Mao, Hao Xiang, Xueru Wen, Zichao Li, Yaojie Lu, Hongyu Lin, Le Sun, Xianpei Han

Implicit Deep Latent Variable Models for Text Generation (D19-1)

Challenge:	Automated Alignment (ALM) is a set of algorithms designed to align Large Language Models (LLMs) with human intentions and values while minimizing manual intervention.
Approach:	They propose an open-source toolkit that integrates mainstream automated algorithms through a consistent interface and an accessible workflow supporting one-click execution for prompt synthesis and automatic alignment signal construction.
Outcome:	The proposed framework enables easy reproduction of existing results through extensive benchmarks and facilitates the development of novel approaches via modular components.

Copied to clipboard

Le Fang, Chunyuan Li, Jianfeng Gao, Wen Dong, Changyou Chen

CoMave: Contrastive Pre-training with Multi-scale Masking for Attribute Value Extraction (2023.findings-acl)

Challenge:	Variational auto-encoders have been used for text generation but their representation power is limited due to two reasons.
Approach:	They advocate sample-based representations of variational distributions for natural language . they further develop an LVM to directly match the aggregated posterior to the prior .
Outcome:	The proposed model can be viewed as a natural extension of VAEs with a regularization of maximizing mutual information, mitigating the "posterior collapse" issue.

Copied to clipboard

Xinnan Guo, Wentao Deng, Yongrui Chen, Yang Li, Mengdi Zhou, Guilin Qi, Tianxing Wu, Dong Yang, Liubin Wang, Yong Pan

PEAR: Planner-Executor Agent Robustness Benchmark (2026.findings-eacl)

Challenge:	Existing methods to extract product features from unstructured text still suffer from problems . e-commerce platforms are focusing on multi-scale values, which can be confusing .
Approach:	They propose a pre-training technique to automatically obtain attribute value pairs from product descriptions to aid e-commerce.
Outcome:	The proposed method improves on the existing token-level masking strategy and achieves state-of-the-art on four benchmarks.

Copied to clipboard

Shen Dong, Mingxuan Zhang, Pengfei He, Li Ma, Bhavani Thuraisingham, Hui Liu, Yue Xing

MPO: Boosting LLM Agents with Meta Plan Optimization (2025.findings-emnlp)

Challenge:	Existing studies examine isolated attack surfaces or specific scenarios, leaving a lack of holistic understanding of MAS vulnerabilities.
Approach:	They propose a benchmark to evaluate the utility and vulnerability of planner–executor MAS.
Outcome:	The proposed benchmark evaluates planner–executor MAS on a widely adopted design.

Copied to clipboard

Weimin Xiong, Yifan Song, Qingxiu Dong, Bingchan Zhao, Feifan Song, null XWang, Sujian Li

When Safety Alignment Fails to Generalize: Probing with Language Game Jailbreaks (2026.findings-acl)

Challenge:	Existing methods for interactive planning tasks suffer from planning hallucinations and require retraining for each new agent.
Approach:	They propose a framework that leverages explicit guidance through meta plans to assist agent planning and enables continuous optimization based on feedback from the agent’s task execution.
Outcome:	The proposed framework outperforms existing baselines on two representative tasks and significantly improves task completion efficiency and generalization capabilities.

Copied to clipboard

Zewen Long, Yu Peng, Fangming Dong, Congyi Li, Xingmao Guan, Shu Wu, Kai Chen

SuLoRA: Subspace Low-Rank Adaptation for Parameter-Efficient Fine-Tuning (2025.findings-acl)

Challenge:	Existing safety alignment methods rely on fixed or narrow transformation schemes to generalize . existing methods based on fixed and narrow transformations are often inadequate .
Approach:	They propose a framework for discovering and refining language game-based jailbreaks to probe alignment generalization.
Outcome:	The proposed framework allows controlled exploration of alignment behavior across closely related linguistic variants.

Copied to clipboard

Chenhao Ding, Jiangyang Li, SongLin Dong, Xinyuan Gao, Yuhang He, Yihong Gong

Boosting Textural NER with Synthetic Image and Instructive Alignment (2024.findings-acl)

Challenge:	Existing methods for fine-tuning large language models (LLMs) introduce parameter interference, leading to a gap in generalization performance for specific tasks compared to full fine-uning.
Approach:	They propose a parameter-separated low-rank adapter to account for task differences by decomposing LoRA’s parameter matrix into multiple independent subspaces and assigning them differentially to distinct tasks.
Outcome:	The proposed method outperforms LoRA in trainable parameter efficiency and overall model performance on various NLP tasks.

Copied to clipboard

Jiahao Wang, Wenjun Ke, Peng Wang, Hang Zhang, Dong Nie, Jiajun Liu, Guozheng Li, Ziyu Shang

Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models (2025.acl-long)

Challenge:	Named entity recognition (NER) is a key task reliant on textual data.
Approach:	They propose a method to transform NER into a multimodal task by using images from the internet as auxiliaries.
Outcome:	The proposed method surpasses all text-only baselines and improves F1 score by 1.4% to 2.3% on prominent MNER datasets.

Copied to clipboard

Zhisong Zhang, Yan Wang, Xinting Huang, Tianqing Fang, Hongming Zhang, Chenlong Deng, Shuaiyi Li, Dong Yu

Can Large Multimodal Models Uncover Deep Semantics Behind Images? (2024.findings-acl)

Challenge:	Large language models have demonstrated remarkable performance across a wide range of language tasks due to their remarkable ability in context modeling.
Approach:	They propose to use parallel context encoding to reduce attention entropy by incorporating attention sinks and selective mechanisms to reduce irregular attention . they also propose to incorporate attention sink mechanisms into the parallel encoded context to reduce the irregular attention.
Outcome:	The proposed methods lower irregular attention entropy and narrow performance gaps.

Copied to clipboard

Yixin Yang, Zheng Li, Qingxiu Dong, Heming Xia, Zhifang Sui

Knowledge Graph-Enhanced Large Language Models via Path Selection (2024.findings-acl)

Challenge:	Existing studies on visual deep semantics focus primarily on superficial description of images, revealing a notable deficiency in the systematic investigation of the inherent deep semantic.
Approach:	They propose a benchmark to assess Large Multimodal Models’ (LMMs) capacities of visual deep semantics.
Outcome:	The proposed benchmark demonstrates a substantial gap between the deep semantic comprehension capabilities of existing LMMs and humans.

Copied to clipboard

Haochen Liu, Song Wang, Yaochen Zhu, Yushun Dong, Jundong Li

TreeRL: LLM Reinforcement Learning with On-Policy Tree Search (2025.acl-long)

Challenge:	Large Language Models (LLMs) have shown unprecedented performance in various real-world applications, but they are known to generate factually inaccurate outputs.
Approach:	They propose a framework to integrate external knowledge extracted from Knowledge Graphs (KGs) they propose to generate scores for knowledge paths with input texts via latent semantic matching.
Outcome:	Experiments on real-world datasets validate the effectiveness of a framework to extract knowledge from Knowledge Graphs (KGs) incorporating external knowledge has become a promising strategy to improve the factual accuracy of LLM-generated outputs.

Copied to clipboard

Zhenyu Hou, Ziniu Hu, Yujiang Li, Rui Lu, Jie Tang, Yuxiao Dong

MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)

Challenge:	Existing methods for On-Policy LLM RL typically train a separate process reward model, which suffers from distribution mismatch and reward hacking.
Approach:	They propose a reinforcement learning framework that directly incorporates on-policy tree search for RL training.
Outcome:	Experiments on math and code reasoning benchmarks show that tree search achieves superior performance compared to traditional ChainRL.

Copied to clipboard

Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li

MC-indexing: Effective Long Document Retrieval via Multi-view Content-aware Indexing (2024.findings-emnlp)

Challenge:	Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities.
Approach:	They propose a comprehensive benchmark covering 29 languages, built on an English benchmark.
Outcome:	The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark.

Copied to clipboard

Kuicai Dong, Derrick Goh Xin Deik, Yi Lee, Hao Zhang, Xiangyang Li, Cong Zhang, Yong Liu

Explaining Graph Neural Networks with Large Language Models: A Counterfactual Perspective on Molecule Graphs (2024.findings-emnlp)

Challenge:	Existing methods for document question answering do not consider content structures, resulting chunks exclude vital information or include irrelevant content.
Approach:	They propose a method that segments document into content chunks and represents each content chunk in raw-text, keywords, and summary views.
Outcome:	The proposed method significantly improves recall of long document question answering datasets compared to state-of-the-art chunking schemes.

Copied to clipboard

Yinhan He, Zaiyi Zheng, Patrick Soga, Yaochen Zhu, Yushun Dong, Jundong Li

Towards Abstractive Grounded Summarization of Podcast Transcripts (2022.acl-long)

Challenge:	Graph Neural Networks (GNNs) are successful in molecular property prediction tasks, but their outputs are often black-box and not easily understandable by humans.
Approach:	They propose a method to unleash the power of large language models (LLMs) to explain GNNs for molecular property prediction.
Outcome:	The proposed method uses autoencoder to generate the counterfactual graph topology from a set of counterfact text pairs based on an input graph.

Copied to clipboard

Kaiqiang Song, Chen Li, Xiaoyang Wang, Dong Yu, Fei Liu

RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery (2025.findings-acl)

Challenge:	Podcast summarization is of practical benefit to content providers and consumers . however, podcast summarizing faces significant challenges including factual inconsistencies . speech recognizers induce transcription errors and abstractive summarisation models may hallucinate .
Approach:	They propose a method to generate podcast summaries while grounding segments in specific regions of the transcript to allow full inspection of summary details.
Outcome:	The proposed method can produce an abstractive summary while grounding segments in specific regions of the transcript to allow full inspection of summary details.

Copied to clipboard

Hongchao Gu, Dexun Li, Kuicai Dong, Hao Zhang, Hang Lv, Hao Wang, Defu Lian, Yong Liu, Enhong Chen

Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models (2024.findings-acl)

Challenge:	Existing methods for knowledge-intensive long texts struggle with issues like hallucinations, topic incoherence, and significant latency.
Approach:	They propose a retrieval-augmented long text generation framework with writing Planning and Information to address these challenges.
Outcome:	The proposed framework outperforms state-of-the-art methods on a freshWiki-2024 dataset.

Copied to clipboard

Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, Ge Li

V-Oracle: Making Progressive Reasoning in Deciphering Oracle Bones for You and Me (2025.acl-long)

Challenge:	Considering the vast size and wide-ranging sources of LLMs’ training data, it could explicitly or implicitly include test data.
Approach:	They propose a Contamination Detection via output Distribution (CDD) which detects data contamination only by identifying the peakedness of LLM's output distribution.
Outcome:	The proposed method improves performance by 21.8%-30.2% on humanEval and TED: trustworthy evaluation via output distribution.

Copied to clipboard

Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Jiapeng Wang, YiFan Zhang, Zhuoma GongQue, Chong Sun, Yida Xu, Yadong Xue, Ye Tian, Zhimin Bao, Lan Yang, Chen Li, Honggang Zhang

AHA: Aligning Large Audio-Language Models for Reasoning Hallucinations via Counterfactual Hard Negatives (2026.findings-acl)

Challenge:	Deciphering oracle bone scripts using AI technology is not an overnight task due to the evolution of written language over millennia.
Approach:	They propose a framework that utilizes Large Multi-modal Models (LMMs) for interpreting Oracle Bone Script (OBS).
Outcome:	The proposed framework provides quantitative analyses and superior deciphering capability.

Copied to clipboard

Yanxi Chen, Wenhui Zhu, Xiwen Chen, Zhipeng Wang, Xin Li, Peijie Qiu, Hao Wang, Xuanzhao Dong, Yujian Xiong, Anderson Schneider, Yuriy Nevmyvaka, Yalin Wang

Syntactic Multi-view Learning for Open Information Extraction (2022.emnlp-main)

Challenge:	Large Audio-Language Models suffer from hallucinations, e.g., generating text not grounded in the audio input.
Approach:	They propose a framework to address hallucination problems in large audio-language models . they use a preference dataset to test the model's accuracy .
Outcome:	The proposed model outperforms the latest SOTA methods in terms of performance and generalization.

Copied to clipboard

Kuicai Dong, Aixin Sun, Jung-Jae Kim, Xiaoli Li

Select Before Use: On the Importance of Reference Model Selection in Preference Alignment (2026.acl-long)

Challenge:	Open Information Extraction (OpenIE) aims to generate structured tuples from unstructured open-domain text.
Approach:	They propose to model constituency and dependency trees into word-level graphs and combine them with sentential semantic representations to extract relational tuples.
Outcome:	The proposed model integrates constituency and dependency trees into word-level graphs and enables neural OpenIE to learn from syntactic structures.

Copied to clipboard

Muyang Li, Runze Wu, Xiangyu Zhao, Bo Han, Daoyi Dong, Tongliang Liu

Challenge:	Supervised Fine-Tuning (SFT) is used as the initialization and reference model for subsequent preference alignment.
Approach:	They propose to use RewardRank to estimate initial implicit alignment between reference model and preference objective to ensure LLMs generate safe, helpful, and instruction-aligned content.
Outcome:	Empirical evidence shows that using the selected model as reference can gain up to 67.6% relative increase on length-controlled win rate compared to baselines.

DemoSG: Demonstration-enhanced Schema-guided Generation for Low-resource Event Extraction (2023.findings-emnlp)

Copied to clipboard

Gang Zhao, Xiaocheng Gong, Xinjie Yang, Guanting Dong, Shudong Lu, Si Li

Beyond ’Aha!’: Toward Systematic Meta-Abilities Alignment in Large Reasoning Models (2026.findings-acl)

Challenge:	Current Event Extraction methods focus on high-resource scenarios, which requires large amount of annotated data.
Approach:	They propose a demonstration-based learning paradigm for EE to fully use annotated data . they propose EE as a natural language generation task guided by schema-based prompts .
Outcome:	The proposed model outperforms current methods in low-resource scenarios.

Copied to clipboard

Zhiyuan Hu, Yibo Wang, Hanze Dong, Yuhui Xu, Amrita Saha, Caiming Xiong, Bryan Hooi, Junnan Li

LPZero: Language Model Zero-cost Proxy Search from Zero (2024.findings-emnlp)

Challenge:	Prior work has shown that outcome-based reinforcement learning (RL) can incidentally elicit advanced reasoning behaviors such as self-correction, backtracking, and verification.
Approach:	They explicitly align large reasoning models with three meta-abilities: deduction, induction, and abduction, using automatically generated, self-verifiable tasks.
Outcome:	The proposed model aligns models with deduction, induction, and abduction meta-abilities using automatically generated, self-verifiable tasks.

Copied to clipboard

Peijie Dong, Lujun Li, Xiang Liu, Zhenheng Tang, Xuebo Liu, Qiang Wang, Xiaowen Chu

Measure Twice, Click Once: Co-evolving Proposer and Visual Critic via Reinforcement Learning for GUI Grounding (2026.acl-long)

Challenge:	Existing zero-cost (ZC) proxies rely on expert knowledge and incur significant trial-and-error costs.
Approach:	They propose a framework that automatically designs zero-cost (ZC) proxies for various tasks and incorporates genetic programming to find the optimal symbolic composition.
Outcome:	The proposed framework achieves higher ranking consistency than human-designed proxies on NLP tasks.

Copied to clipboard

Wenkai Wang, Xiyun Li, Hongcan Guo, Wenhao Yu, Tianqing Fang, Haitao Mi, Dong Yu, Shengyu Zhang

R-Judge: Benchmarking Safety Risk Awareness for LLM Agents (2024.findings-emnlp)

Challenge:	Graphical User Interface (GUI) grounding requires mapping natural language instructions to precise pixel coordinates due to visually homogeneous elements and dense layouts.
Approach:	They propose to replace static consistency strategies with a learnable selection mechanism that selects the optimal target by critiquing its own proposals rendered on the screenshot.
Outcome:	The proposed model significantly improves both grounding and critiquing capabilities over 6 benchmarks.

Copied to clipboard

Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, Rui Wang, Gongshen Liu

Challenge:	Large language models (LLMs) have shown compelling abilities in reasoning, decision-making, and instruction following.
Approach:	They propose a benchmark to evaluate the proficiency of large language models (LLMs) in judging and identifying safety risks given agent interaction records.
Outcome:	The proposed model outperforms the best-performing model, GPT-4o, while no other models significantly exceed the random.

Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts (2026.acl-long)

Copied to clipboard

Di Zhang, Xun Wu, Shaohan Huang, Lingjie Jiang, Yaru Hao, Li Dong, Zewen Chi, Zhifang Sui, Furu Wei

DRA-GRPO: Your GRPO Needs to Know Diverse Reasoning Paths for Mathematical Reasoning (2026.findings-acl)

Challenge:	Reinforcement learning with verifiable rewards (RLVR) training with Mixture-of-Experts policies remains fragile and prone to reward collapse.
Approach:	They propose a router shift-based policy optimization method that computes a per-token router-shift ratio conditioned on the previously activated experts and applies stop-gradient and a lower-bound floor.
Outcome:	The proposed method achieves better performance and greater stability than previous methods.

Copied to clipboard

Xiwen Chen, Wenhui Zhu, Peijie Qiu, Xuanzhao Dong, Hao Wang, Haiyu Wu, Huayu Li, Aris Sotiras, Yalin Wang, Abolfazl Razi

Does Your Model Classify Entities Reasonably? Diagnosing and Mitigating Spurious Correlations in Entity Typing (2022.emnlp-main)

Challenge:	Existing methods for group-relative policy optimization rely on scalar correctness rewards that are often non-injective with respect to semantic content.
Approach:	They propose a framework that calibrates the reward signal using the semantic density of sampled groups.
Outcome:	The proposed framework outperforms strong baselines on five math benchmarks with 7,000 samples and 55 cost.

Copied to clipboard

Nan Xu, Fei Wang, Bangzheng Li, Mingtao Dong, Muhao Chen

FoldMoE: Efficient Long Sequence MoE Training via Attention-MoE Pipelining (2025.acl-long)

Challenge:	Existing entity typing models are subject to spurious correlations due to shortcuts and biased training.
Approach:	They propose a method to augment existing model biases by combining spurious correlations with debiasedcounterparts to improve generalization.
Outcome:	The proposed method improves generalization of different entity typing models on the original and debiased test sets.

Copied to clipboard

Guichao Zhu, Lintian Lei, Yuhao Qing, Yichao Fu, Fanxin Li, Dong Huang, Zekai Sun, Heming Cui

FlowEval: A Consensus-Based Dialogue Evaluation Framework Using Segment Act Flows (2022.emnlp-main)

Challenge:	Existing approaches to training LLMs with Mixture-of-Experts (MoE) architecture on long sequences are limited by the insufficient computation.
Approach:	They propose a MoE training system that enables token-level overlapping across entire Transformer blocks through novel attention-MoE pipelining.
Outcome:	The proposed system achieves 1.49x and 2.72x speedup over state-of-the-art token-level overlapping and non-overlapping baselines on GPT-MoE models with sequences up to 32K tokens.

Copied to clipboard

Jianqiao Zhao, Yanyang Li, Wanyu Du, Yangfeng Ji, Dong Yu, Michael Lyu, Liwei Wang

EconProver: Towards More Economical Test-Time Scaling for Automated Theorem Proving (2026.acl-long)

Challenge:	Despite recent progress in dialogue evaluation, how to develop automatic metrics remains an open problem.
Approach:	They propose a consensus-based framework for dialog evaluation using segment act flows . they propose to crowdsource a large-scale dataset for it to be evaluated .
Outcome:	The proposed framework can reach the best or comparable correlation with human evaluation.

Copied to clipboard

Mukai Li, Linfeng Song, Zhenwen Liang, Jiahao Xu, Shansan Gong, Qi Liu, Haitao Mi, Dong Yu

Beyond the Safety Tax: Mitigating Unsafe Text-to-Image Generation via External Safety Rectification (2026.findings-acl)

Challenge:	Large Language Models (LLMs) have recently advanced the field of Automated Theorem Proving (ATP) Existing cost analyses regulate only the number of sampling passes, ignoring the substantial disparities in sampling costs.
Approach:	They propose to integrate two complementary methods into a unified EconRL pipeline to increase pass rates under constrained sampling passes.
Outcome:	The proposed method reduces token usage and sample passes while maintaining the original performance.

Copied to clipboard

Xiangtao Meng, Yingkai Dong, Ning Yu, Li Wang, Zheng Li, Shanqing Guo

ReasMark: A Robust Watermark for Attributing LLM Reasoning Under Knowledge Distillation Attacks (2026.acl-long)

Challenge:	Existing safety defenses typically intervene internally within the generative model, but suffer from severe concept entanglement, leading to degradation of benign generation quality.
Approach:	They propose a structurally isolated safety module that performs external, interpretable rectification without modifying the base model.
Outcome:	The proposed module performs external, interpretable rectification without modifying the base model.

Copied to clipboard

Peizhuo Lv, Ruihua Zhou, Yunpeng Li, Ruigang Liang, Xingshuo Han, XiaoFeng Wang, Wei Dong, Yuling Liu

Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement (2025.acl-long)

Challenge:	Existing reasoning-enhanced large language models fail to provide reliable attribution of reasoning behavior once it is transferred through knowledge distillation.
Approach:	They propose to embed a reasoning-length gap in a model by querying a target domain and training a local student to imitate its outputs.
Outcome:	et al. show that ReasMark outperforms baselines while preserving task utility.

Copied to clipboard

Yichen Dong, Xinglin Lyu, Junhui Li, Daimeng Wei, Min Zhang, Shimin Tao, Hao Yang

Machine Translation

Challenge:	Recent research has shown that large language models (LLMs) can enhance translation quality through self-refinement.
Approach:	They propose to extend translation refinement from sentence-level to document-level by using document-to-document (Doc2Doc) translations.
Outcome:	The proposed method improves translation quality across ten translation tasks with LLaMA-3-8B-Instruct and Mistral-Nemo-Instru.

Better Zero-Shot Reasoning with Role-Play Prompting (2024.naacl-long)

Copied to clipboard

Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, Xiaohang Dong

Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding (2024.findings-acl)

Challenge:	Recent years have witnessed a paradigm shift in natural language processing, driven by large language models such as GPT-3, PaLM, and Llama.
Approach:	They propose a strategy for role-play prompting and assess its performance under the zero-shot setting.
Outcome:	The proposed method outperforms the standard zero-shot prompting approach across 12 reasoning benchmarks.

Copied to clipboard

Heming Xia, Zhe Yang, Qingxiu Dong, Peiyi Wang, Yongqi Li, Tao Ge, Tianyu Liu, Wenjie Li, Zhifang Sui

Fine-Grained Entity Typing via Hierarchical Multi Graph Convolutional Networks (D19-1)

Challenge:	Large Language Models (LLMs) have a high inference latency stemming from autoregressive decoding.
Approach:	They propose a novel decoding paradigm that drafts multiple tokens and verifies them in parallel . they aim to provide a catalyst for further research on Speculative Decoding .
Outcome:	The proposed method drafts multiple tokens and verifies them in parallel . it can be used to accelerate inference in large language models.

Copied to clipboard

Hailong Jin, Lei Hou, Juanzi Li, Tiansi Dong

MGPO: Thinking with Images via Multi-Turn Grounding-Based Reinforcement Learning (2026.findings-acl)

Challenge:	Existing methods for inferring the fine-grained type of an entity from knowledge base are incomplete and lack type information.
Approach:	They propose a novel Deep Learning architecture to infer the fine-grained type of an entity from a knowledge base.
Outcome:	The proposed method significantly outperforms four state-of-the-art methods on two large-scale datasets.

Copied to clipboard

Xinyu Huang, Yuhao Dong, Weiwei Tian, Bo Li, Rui Feng, Ziwei Liu

Beyond A Single AI Cluster: A Survey of Decentralized LLM Training (2025.emnlp-main)

Challenge:	State-of-the-art large multimodal models face challenges when processing high-resolution images, as these inputs are converted into enormous visual tokens, many of which are irrelevant to the downstream task.
Approach:	They propose a multi-turn grounding-based policy optimization framework that enables LMMs to iteratively focus on key visual regions by automatically cropping sub-images based on model-predicted grounding coordinates within a multiple-turn conversation framework.
Outcome:	The proposed framework improves on Qwen2.5-VL-7B with 21K samples and surpasses OpenAI’s o1 and GPT-4o models on the out-of-distribution (OOD) V* Bench.

Copied to clipboard

Haotian Dong, Jingyan Jiang, Rongwei Lu, Jiajun Luo, Jiajun Song, Bowen Li, Ying Shen, Zhi Wang

Text Clustering

Challenge:	Decentralized LLM training leverages dispersed resources at varying scales.
Approach:	They propose a resource-driven paradigm that leverages dispersed resources across clusters, datacenters and even regions.
Outcome:	The proposed model scales are 175 billion to 660 billion parameters, and the exponential growth in computational requirements poses significant challenges.

Scaling Laws for Linear Complexity Language Models (2024.emnlp-main)

Copied to clipboard

Xuyang Shen, Dong Li, Ruitao Leng, Zhen Qin, Weigao Sun, Yiran Zhong

Graph Counselor: Adaptive Graph Exploration via Multi-Agent Synergy to Enhance LLM Reasoning (2025.acl-long)

Challenge:	Existing scaling laws for large language models are unclear, but they are useful for scalability.
Approach:	They propose scaling laws for linear complexity language models to establish a foundation for their scalability.
Outcome:	The proposed models demonstrate superior linguistic proficiency and knowledge retention.

Copied to clipboard

Junqi Gao, Xiang Zou, Ying Ai, Dong Li, Yichen Niu, Biqing Qi, Jianxing Liu

ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding (2026.findings-acl)

Challenge:	Existing methods for enhancing LLM reliability suffer from inefficient information aggregation and rigid reasoning schemes.
Approach:	They propose a method that explicitly models external knowledge integration capabilities by explicitly modeling knowledge relationships.
Outcome:	The proposed method outperforms existing methods in multiple graph reasoning tasks.

Copied to clipboard

Tian Xueyun, Wei Li, Bingbing Xu, Heng Dong, Yuanzhuo Wang, Huawei Shen

Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study (2023.emnlp-main)

Challenge:	Existing omni-multimodal large language models lack incomplete modality support or lack autonomous proactive monitoring.
Approach:	They propose a real-time omni-multimodal assistant for unified reactive and proactive interaction that decouples response initiation from generation to ensure precise triggering without task conflict.
Outcome:	The proposed model achieves state-of-the-art performance on proactive tasks while competing in reactive settings.

Copied to clipboard

Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro

DL-QAT: Weight-Decomposed Low-Rank Quantization-Aware Training for Large Language Models (2024.emnlp-industry)

Challenge:	a recent study shows that retrieval-augmented LMs can improve text generation quality and accuracy.
Approach:	They propose a model that reproduces RETRO parameters while retrieving a text corpus . they find RETRO outperforms GPT on text generation with less repetition .
Outcome:	The proposed model outperforms standard retrieval-augmented GPT and retrieval augmented GTP on text generation and accuracy tasks.

Copied to clipboard

Wenjing Ke, Zhe Li, Dong Li, Lu Tian, Emad Barsoum

Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning (2025.acl-long)

Challenge:	Quantization-aware Training (QAT) is a popular technique for reducing memory usage and improving computational efficiency in large language models.
Approach:	They propose a weight-decomposed low-rank quantization-aware training approach that integrates QAT with a group-specific quantization magnitude adjustment.
Outcome:	The proposed method outperforms the state-of-the-art method on LLaMA and LLama2 models.

Copied to clipboard

Hui Liu, Wenya Wang, Hao Sun, Chris Xing Tian, Chenqi Kong, Xin Dong, Haoliang Li

Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning (2023.acl-long)

Challenge:	Recent learning-based demonstration selection methods have proven beneficial to in-context learning (ICL) by choosing more useful exemplars.
Approach:	They propose two methods to capture task-agnostic similarities between input and output of LLMs.
Outcome:	The proposed methods integrate task-agnostic similarities of different levels between input and output of exemplars and test cases to eliminate costly data collection.

Copied to clipboard

Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song

AdaPrompt: Adaptive Model Training for Prompt-based NLP (2022.findings-emnlp)

Challenge:	XY-LENT: X-Y bitext enhanced Language ENcodings achieves state-of-the-art performance over 5 cross-lingual tasks within all model size bands.
Approach:	They propose a method for building multilingual representation models that are competitive with existing models and more parameter efficient.
Outcome:	The proposed model outperforms XLM-R XXL and is 5x and 6x smaller respectively.

Copied to clipboard

Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang

Crafting Customisable Characters with LLMs: A Persona-Driven Role-Playing Agent Framework (2025.findings-emnlp)

Challenge:	Prompt-based learning can tackle zero-shot and few-shot NLP tasks . authors propose a method that makes use of pre-trained language models .
Approach:	They propose to map NLP tasks into natural language prompts, which are then filled by pre-trained language models.
Outcome:	The proposed method outperforms standard prompt-based methods in few-shot settings.

Copied to clipboard

Bohao Yang, Dong Liu, Chenghao Xiao, Kun Zhao, Chen Tang, Chao Li, Lin Yuan, Yang Guang, Chenghua Lin

Challenge:	Large Language Models (LLMs) are capable of generating human-like text, but the potential for freely customisable characters remains underexplored.
Approach:	They propose a framework which employs Large Language Models to create freely customisable characters through personalised characteristic feature injection.
Outcome:	The proposed framework provides valuable insights for developing more accurate and customisable human simulacra.

LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding (2024.acl-long)

Copied to clipboard

Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, Juanzi Li

Adaptive Tool Use in Large Language Models with Meta-Cognition Trigger (2025.acl-long)

Challenge:	Large language models (LLMs) can only handle texts a few thousand tokens long, limiting their applications on longer sequence inputs, such as books, reports, and codebases.
Approach:	They propose a bilingual, multi-task benchmark for long context understanding that extends context windows and more sophisticated memory mechanisms to improve models' long context capabilities.
Outcome:	The proposed model outperforms open-source models but struggles on longer contexts.

Copied to clipboard

Wenjun Li, Dexun Li, Kuicai Dong, Cong Zhang, Hao Zhang, Weiwen Liu, Yasheng Wang, Ruiming Tang, Yong Liu

Uncertainty Propagation on LLM Agent (2025.acl-long)

Challenge:	Existing research expands the tool arrays of large language models (LLMs), but the necessity of using these tools is often overlooked, leading to indiscriminate tool invocation.
Approach:	They propose a meta-cognition proxy proxy for LLMs self-assessment of their capabilities, reflecting the model’s awareness of its own limitations.
Outcome:	The proposed strategy is fine-tuned-free and costs minimal.

Copied to clipboard

Qiwei Zhao, Dong Li, Yanchi Liu, Wei Cheng, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Huaxiu Yao, Chen Zhao, Haifeng Chen, Xujiang Zhao

Challenge:	Existing methods for estimating uncertainty in large language models (LLMs) focus on final-step outputs, which fail to account for cumulative uncertainty over multi-step decision-making process and dynamic interactions between agents and their environments.
Approach:	They propose a framework that propagates uncertainty through each step of an LLM-based agent’s reasoning process.
Outcome:	Extensive experiments on benchmark datasets show that the proposed framework outperforms state-of-the-art methods by 20%.

Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers (2023.findings-emnlp)

Copied to clipboard

Weng Tam, Xiao Liu, Kaixuan Ji, Lilong Xue, Jiahua Liu, Tao Li, Yuxiao Dong, Jie Tang

More than Text: Multi-modal Chinese Word Segmentation (2021.acl-short)

Challenge:	Prompt tuning is a technique that updates few parameters in pre-trained models for language understanding and generation tasks.
Approach:	They propose to leverage prompt tuning for neural text retrieval to improve generalization and cross-domain generalization.
Outcome:	The proposed approach can mitigate the two issues faced by fine-tuning retrieval methods and improve the out-of-domain zero-shot generalization of the retrieval models.

Copied to clipboard

Dong Zhang, Zheng Hu, Shoushan Li, Hanqian Wu, Qiaoming Zhu, Guodong Zhou

SPACE-2: Tree-Structured Semi-Supervised Contrastive Pre-training for Task-Oriented Dialog Understanding (2022.coling-1)

Challenge:	Currently, word segmentation is performed in many languages without word delimiters.
Approach:	They propose to combine the multi-modality to perform Chinese word segmentation . they propose a time-dependent multi-module interactive model to integrate multi-modality information .
Outcome:	The proposed model integrates multi-modal information for word sequence labeling with Chinese language as target . the proposed model performs well on three training sets on Chinese and other languages without word delimiters.

Copied to clipboard

Wanwei He, Yinpei Dai, Binyuan Hui, Min Yang, Zheng Cao, Jianbo Dong, Fei Huang, Luo Si, Yongbin Li

KoCo-Bench: Can Large Language Models Leverage Domain Knowledge in Software Development? (2026.acl-long)

Challenge:	Existing methods for dialog understanding only consider self-augmented dialogs as positive samples and treat all other dialogs like negative ones.
Approach:	They propose a tree-structured pre-trained conversation model which learns dialog representations from limited labeled dialogs and large-scale unlabeled dialog corpora via semi-supervised contrastive pre-training.
Outcome:	The proposed model can achieve state-of-the-art results on the DialoGLUE benchmark.

Copied to clipboard

Xue Jiang, Ge Li, Jiaru Qian, Xianjie Shi, Chenjie Li, Hao Zhu, Ziyu Wang, Jielun Zhang, Zeyu Zhao, Kechi Zhang, Jia Li, Wenpin Jiao, Zhi Jin, Yihong Dong

Multi-modal Multi-label Emotion Detection with Modality and Label Dependence (2020.emnlp-main)

Challenge:	Existing domain-specific code benchmarks focus on assessing what knowledge LLMs possess rather than how they acquire and apply new knowledge.
Approach:	They propose a benchmark to evaluate domain specialization methods in real-world software development.
Outcome:	KOCO-bench is a new benchmark for evaluating domain specialization methods in real-world software development.

Copied to clipboard

Dong Zhang, Xincheng Ju, Junhui Li, Shoushan Li, Qiaoming Zhu, Guodong Zhou

LongAlign: A Recipe for Long Context Alignment of Large Language Models (2024.findings-emnlp)

Challenge:	Existing studies on multi-label emotion detection focus on one modality . current studies focus on label dependence, but there is no consensus on the model .
Approach:	They propose a multi-modal sequence-to-set approach to model label dependence and modality dependence in a multiple-modal scenario.
Outcome:	The proposed approach is able to model the label dependence and the modality dependence in a multi-modal scenario.

Copied to clipboard

Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, Juanzi Li

Confidence Estimation for LLMs in Multi-turn Interactions (2026.findings-acl)

Challenge:	Existing studies to build long context language models focus on context extension and continual training on long text.
Approach:	They propose a recipe for instruction fine-tuning on input sequences of similar length . they adopt packing and sorted batching strategies to speed up supervised fine-uning .
Outcome:	The proposed model outperforms existing recipes for LLMs in long context tasks by 30% while maintaining proficiency in handling short, generic tasks.

Copied to clipboard

Caiqi Zhang, Ruihan Yang, Xiaochen Zhu, Chengzu Li, Tiancheng Hu, Yijiang River Dong, Deqing Yang, Nigel Collier

FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models (2025.emnlp-main)

Challenge:	Despite recent progress, most prior work studies confidence in single-turn question answering.
Approach:	They propose a logit-based probe that measures confidence in multi-turn dialogues . they propose 'infoECE' and a "hinter-guesser" paradigm for generating controlled evaluations based on data .
Outcome:	The proposed framework is grounded in calibration and monotonicity of confidence as more information becomes available.

Copied to clipboard

Hengxing Cai, Jinhan Dong, Jingjun Tan, Jingcheng Deng, Sihang Li, Zhifeng Gao, Haidong Wang, Zicheng Su, Agachai Sumalee, Renxin Zhong

A Survey on Open Information Extraction from Rule-based Model to Large Language Model (2024.findings-emnlp)

Challenge:	Existing methods for vision-and-language navigation struggle with insufficient multimodal fusion, weak generalization, and poor interpretability.
Approach:	They propose a framework for UAV vision-and-language navigation that integrates natural language instructions with visual observations to improve multimodal fusion and interpretability.
Outcome:	The proposed framework achieves state-of-the-art performance across all scenarios, with a 9.22% higher success rate than the strongest baseline in unseen environments.

Copied to clipboard

Liu Pai, Wenyang Gao, Wenjie Dong, Lin Ai, Ziwei Gong, Songfang Huang, Li Zongsheng, Ehsan Hoque, Julia Hirschberg, Yue Zhang

How Far are LLMs from Being Our Digital Twins? A Benchmark for Persona-Based Behavior Chain Simulation (2025.findings-acl)

Challenge:	Open Information Extraction (OpenIE) is a key NLP task aimed at extracting structured information from unstructured text sources.
Approach:	They propose to categorize OpenIE into rule-based, neural, and pre-trained large language models and discuss each within a chronological framework.
Outcome:	The paper categorizes OpenIE approaches into rule-based, neural, and pre-trained large language models, discussing each within a chronological framework.

Copied to clipboard

Rui Li, Heming Xia, Xinfeng Yuan, Qingxiu Dong, Lei Sha, Wenjie Li, Zhifang Sui

Challenge:	Recent studies have focused on dialogue simulation while overlooking human behavior simulation, which is crucial for digital twins.
Approach:	They propose to integrate persona metadata into LLMs and use it to iteratively infer contextually appropriate behaviors within dynamic scenarios.
Outcome:	The proposed model is based on 15,846 distinct behaviors across 1,001 unique personas and incorporates persona metadata to iteratively infer appropriate behaviors within dynamic scenarios.

TinyAlign: Boosting Lightweight Vision-Language Models by Mitigating Modal Alignment Bottlenecks (2026.findings-acl)

Copied to clipboard

Yuanze Hu, Xinyu Wang, Zhichao Yang, Gen Li, Ye Qiu, Zhaoxin Fan, Yifan Sun, Wenjun wu, Jin Dong, Xiaotie Deng

Capturing Conversational Interaction for Question Answering via Global History Reasoning (2022.findings-naacl)

Challenge:	Lightweight Vision-Language Models (VLMs) are indispensable for resource-constrained applications.
Approach:	They propose a framework that retrieves context from a memory bank to enhance alignment . they propose EMI-based approach to align vision and language models .
Outcome:	The proposed framework reduces training loss, accelerates convergence, and enhances task performance with negligible computational overhead.

Copied to clipboard

Jin Qian, Bowei Zou, Mengxing Dong, Xiao Li, AiTi Aw, Yu Hong

SMR: State Memory Replay for Long Sequence Modeling (2024.findings-acl)

Challenge:	Existing studies have studied history-dependent reasoning for question answering . utilizing global conversation history for enhancement is gaining interest .
Approach:	They propose to establish long-distance dependency among global utterances in multi-turn conversation.
Outcome:	The proposed method improves on QuAC by 1%, yielding the F1 score of 73.7%.

Copied to clipboard

Biqing Qi, Junqi Gao, Kaiyan Zhang, Dong Li, Jianxing Liu, Ligang Wu, Bowen Zhou

Challenge:	Existing state space models (SSMs) address non-uniform sampling, but their recursive structures impede efficient SSM computation via convolution.
Approach:	They propose a plug-and-play mechanism to solve the Non-Stable State problem by adjusting input sequences with early memories.
Outcome:	The proposed method overcomes the non-uniform sample processing problem . it can achieve Sampling Step Adaptation (SSA) by adjusting input sequences with early memories.

A Silver Bullet or a Compromise for Full Attention? A Comprehensive Study of Gist Token-based Context Compression (2025.acl-long)

Copied to clipboard

Chenlong Deng, Zhisong Zhang, Kelong Mao, Shuaiyi Li, Xinting Huang, Dong Yu, Zhicheng Dou

Challenge:	gist-based context compression methods can achieve only slight performance loss on tasks like retrieval-augmented generation and long-document QA, but it faces challenges in tasks like synthetic recall.
Approach:	They propose two strategies to improve gist-based context compression in large language models.
Outcome:	The proposed methods can achieve only slight performance loss on retrieval-augmented generation and long-document QA tasks, but they face challenges in tasks like synthetic recall.

Variational Graph Autoencoding as Cheap Supervision for AMR Coreference Resolution (2022.acl-long)

Copied to clipboard

Irene Li, Linfeng Song, Kun Xu, Dong Yu

Learning to Sample Replacements for ELECTRA Pre-Training (2021.findings-acl)

Challenge:	Abstract Meaning Representation (AMR) is a way to preserve the semantic meaning of a sentence in a graph.
Approach:	They propose a general pretraining method that leverages any general AMR corpus and even automatically parses AMR data to achieve performance gains of up to 6% absolute F1 points.
Outcome:	The proposed model significantly improves on the previous state-of-the-art model by up to 11% F1.

Copied to clipboard

Yaru Hao, Li Dong, Hangbo Bao, Ke Xu, Furu Wei

Enhancing Multimodal Continual Instruction Tuning with BranchLoRA (2025.acl-long)

Challenge:	Experimental results show that ELECTRA pretrains a discriminator to detect replaced tokens . despite compelling performance, there is no direct feedback loop from discriminator and generator to generator, making replacements biased to correct tokens.
Approach:	They propose to augment sampling with a hardness prediction mechanism to encourage the discriminator to learn what it has not acquired.
Outcome:	The proposed method improves ELECTRA pre-training on various downstream tasks.

Copied to clipboard

Duzhen Zhang, Yong Ren, Zhong-Zhi Li, Yahan Yu, Jiahua Dong, Chenxing Li, Zhilong Ji, Jinfeng Bai

A Length-Extrapolatable Transformer (2023.acl-long)

Challenge:	Existing approaches to fine tune Multimodal Large Language Models (MLLMs) are prone to Catastrophic Forgetting (CF) existing approaches rely on the Mixture-of-Experts (MoE) LoRA framework to preserve previous instruction alignments.
Approach:	They propose an asymmetric tuning-freezing mechanism to mitigate parameter inefficiency . branch-specific routers are introduced to ensure optimal branch distribution over time .
Outcome:	The proposed framework outperforms existing frameworks on the latest MCIT benchmarks.

Copied to clipboard

Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei

LLMBox: A Comprehensive Library for Large Language Models (2024.acl-demos)

Challenge:	Existing Transformers can only deal with the in-distribution size of inputs.
Approach:	They propose a relative position embedding to explicitly maximize attention resolution . they also use blockwise causal attention during inference for better resolution a .
Outcome:	The proposed model achieves strong performance in interpolation and extrapolation settings.

Copied to clipboard

Tianyi Tang, Hu Yiwen, Bingqian Li, Wenyang Luo, ZiJing Qin, Haoxiang Sun, Jiapeng Wang, Shiyi Xu, Xiaoxue Cheng, Geyang Guo, Han Peng, Bowen Zheng, Yiru Tang, Yingqian Min, Yushuo Chen, Jie Chen, Ranchi Zhao, Luran Ding, Yuhao Wang, Zican Dong, Xia Chunxuan, Junyi Li, Kun Zhou, Xin Zhao, Ji-Rong Wen

Visualizing and Understanding the Effectiveness of BERT (D19-1)

Challenge:	a library to facilitate the development, use, and evaluation of large language models (LLMs) is presented.
Approach:	They propose a unified library to facilitate the development, use and evaluation of large language models (LLMs).
Outcome:	The proposed library is based on extensive experiments in a variety of evaluation settings.

Copied to clipboard

Yaru Hao, Li Dong, Furu Wei, Ke Xu

Consistency Regularization for Cross-Lingual Fine-Tuning (2021.acl-long)

Challenge:	Language model pre-training, such as BERT, has achieved strong performance in many NLP tasks.
Approach:	They propose to visualize loss landscapes and optimization trajectories of fine-tuning BERT on specific datasets.
Outcome:	The proposed model improves performance and generalization capability across tasks.

Copied to clipboard

Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

Mitigating Language Confusion through Inference-time Intervention (2025.coling-main)

Challenge:	Experimental results show that consistency regularization improves cross-lingual fine-tuning . pre-trained cross-linguistic models can transfer task-specific supervision from one language to the other .
Approach:	They propose to improve cross-lingual fine-tuning with consistency regularization . they use example consistency regularized to penalize prediction sensitivity to four types of data augmentations .
Outcome:	The proposed method improves cross-lingual fine-tuning across tasks . it can be generalized to other target languages without additional training .

Copied to clipboard

Xie Yunfan, Lixin Zou, Dan Luo, Min Tang, Chenliang Li, Xiangyang Luo, Liming Dong

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment (2021.acl-long)

Challenge:	Existing methods to address the problem of language confusion are incontext learning and supervised fine-tuning (SFT) however, they consume context window space and require extensive data collection.
Approach:	They propose a language-sensitive intervention that detects and assesses language confusion without additional complex mechanisms.
Outcome:	The proposed method detects language confusion and assesses content quality without additional complex mechanisms.

Copied to clipboard

Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers (2021.findings-acl)

Challenge:	Experimental results show that denoising word alignment improves cross-lingual transferability . most applications and resources are still English-centric, making non-English users hard to access.
Approach:	They propose to denoise word alignment as a cross-lingual pre-training task . they first self-label word alignments for parallel sentences and then mask tokens .
Outcome:	The proposed model improves cross-lingual transferability on token-level tasks, especially on question answering, and structured prediction.

Copied to clipboard

Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains (2021.findings-acl)

Challenge:	Existing work on deep self-attention distillation for natural language processing tasks is limited by computational resources and latency.
Approach:	They generalize deep self-attention distillation in MINILM by using only self- attention relation distillation for taskagnostic compression of pretrained Transformers.
Outcome:	The proposed model outperforms the state-of-the-art in a multilingual and multilingual teacher model.

Copied to clipboard

Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei

Multi-Task Neural Model for Agglutinative Language Translation (2020.acl-srw)

Challenge:	Large pre-trained models suffer from domain shift and are not optimal for specific domains.
Approach:	They propose a general approach to developing small, fast and effective pretrained models for specific domains by adapting off-the-shelf general pretrained model and performing task-agnostic knowledge distillation in target domains.
Outcome:	The proposed approach achieves better performance over the BERT BASE model in domain-specific tasks while 3.3 smaller and 5.1 faster than the BRT BASE.

Copied to clipboard

Yirong Pan, Xiao Li, Yating Yang, Rui Dong

KC-GenRe: A Knowledge-constrained Generative Re-ranking Method Based on Large Language Models for Knowledge Graph Completion (2024.lrec-main)

Challenge:	Neural machine translation (NMT) has been gaining popularity in high-resource translation tasks, but struggles in low-ressource and morphologically-rich scenarios.
Approach:	They propose a multi-task neural model that jointly learns to perform bi-directional translation and agglutinative language stemming.
Outcome:	The proposed model can significantly improve translation performance on agglutinative languages by using a small amount of monolingual data.

Copied to clipboard

Yilin Wang, Minghao Hu, Zhen Huang, Dongsheng Li, Dong Yang, Xicheng Lu

GammaE: Gamma Embeddings for Logical Queries on Knowledge Graphs (2022.emnlp-main)

Challenge:	Knowledge graph completion (KGC) is a critical task to predict missing facts among entities.
Approach:	They propose a knowledge-constrained generative re-ranking method based on generative large language models for KGC that can predict missing facts among entities.
Outcome:	The proposed method achieves state-of-the-art performance on four datasets and 9.0% and 11.1% compared to the previous methods.

Copied to clipboard

Dong Yang, Peijun Qing, Yang Li, Haonan Lu, Xiaodong Lin

UnifiedVisual: A Framework for Constructing Unified Vision-Language Datasets (2025.emnlp-main)

Challenge:	Existing methods for embedding knowledge graphs are difficult due to complicated query structures and incomplete graph data.
Approach:	They propose a probabilistic embedding model for encoding entities and queries to answer different types of FOL queries on KGs.
Outcome:	The proposed model outperforms state-of-the-art models on public benchmarks on three large logical query datasets.

Copied to clipboard

Pengyu Wang, Shaojun Zhou, Chenkun Tan, Xinghao Wang, Wei Huang, Zhen Ye, Zhaowei Li, Botian Jiang, Dong Zhang, Xipeng Qiu

Low-Confidence Gold: Refining Low-Confidence Samples for Efficient Instruction Tuning (2025.findings-emnlp)

Challenge:	Existing datasets address understanding and generation in isolation, limiting the performance of unified vision large language models.
Approach:	They propose a dataset that facilitates mutual enhancement between multimodal understanding and generation.
Outcome:	The proposed framework integrates diverse visual and textual inputs and outputs, enabling comprehensive cross-modal reasoning and precise text-to-image alignment.

Copied to clipboard

Hongyi Cai, Jie Li, Mohammad Mahdinur Rahman, Wenzhen Dong

MM-LLMs: Recent Advances in MultiModal Large Language Models (2024.findings-acl)

Challenge:	Low-Confidence Gold (LCG) is a new filtering framework for Large Language Models that curates high-quality subsets while preserving data diversity.
Approach:	They propose a new filtering framework that employs centroid-based clustering and confidence-guided selection for identifying valuable instruction pairs.
Outcome:	The proposed framework improves performance on a subset of 6K samples while maintaining data diversity.

Copied to clipboard

Duzhen Zhang, Yahan Yu, Jiahua Dong, Chenxing Li, Dan Su, Chenhui Chu, Dong Yu

CodeDPO: Aligning Code Models with Self Generated and Verified Source Code (2025.acl-long)

Challenge:	MultiModal Large Language Models (MM-LLMs) have undergone significant advances in the past year . traditional MM models incur substantial computational costs, especially when trained from scratch .
Approach:	They propose a taxonomy encompassing 126 MM-LLMs and summarize key training recipes to enhance their potency.
Outcome:	The proposed models preserve the reasoning and decision-making capabilities of LLMs and empower diverse range of MM tasks.

Copied to clipboard

Kechi Zhang, Ge Li, Yihong Dong, Jingjing Xu, Jun Zhang, Jing Su, Yongfei Liu, Zhi Jin

Open Information Extraction via Chunks (2023.emnlp-main)

Challenge:	Existing training methods for code generation do not improve code correctness and efficiency.
Approach:	They propose a framework that integrates preference learning into code generation to improve code correctness and efficiency.
Outcome:	The proposed framework improves code correctness and efficiency by integrating preference learning into code generation.

Copied to clipboard

Kuicai Dong, Aixin Sun, Jung-jae Kim, Xiaoli Li

Chain-of-Thought as a Lens: Evaluating Structured Reasoning Alignment between Human Preferences and Large Language Models (2026.acl-long)

Challenge:	Existing OIE systems split a sentence into tokens and recognize token spans as tuple relations and arguments.
Approach:	They propose to split a sentence into tokens and recognize token spans as tuple relations and arguments.
Outcome:	The proposed model achieves state-of-the-art on multiple OIE datasets showing that SaC has better properties than sentence as token sequence.

Copied to clipboard

Boxuan Wang, Zhuoyun Li, Xinmiao Huang, Xiaowei Huang, Yi Dong

AIMMerging: Adaptive Iterative Model Merging Using Training Trajectories for Language Model Continual Learning (2025.emnlp-main)

Challenge:	Empirical results show misalignment at greater reasoning depths is driven mainly by alignment errors such as thematic shift and redundant reasoning.
Approach:	They propose a method to quantitatively assess the alignment between multi-step, structured reasoning in large language models and human preferences by constructing semantic-entropy-based matrices over intermediate steps and measuring their divergence.
Outcome:	The proposed method shows that it is consistent with previous studies and can be used as a diagnostic signal.

Copied to clipboard

Yujie Feng, Jian Li, Xiaoyu Dong, Pengfei Xu, Xiaohui Zhou, Yujia Zhang, Zexin Lu, Yasha Wang, Alan Zhao, Xu Chu, Xiao-Ming Wu

Enhancing One-Shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism (2025.coling-main)

Challenge:	Recent model merging-based methods struggle to effectively manage the trade-off between learning new knowledge and preventing catastrophic forgetting.
Approach:	They propose a model merging framework that utilizes learning and forgetting signals from the training trajectory to dynamically monitor the model’s training status.
Outcome:	The proposed framework achieves significant performance improvements over existing state-of-the-art methods on three CL benchmarks with various model sizes (from 770M to 13B).

Copied to clipboard

Guanchen Li, Xiandong Zhao, Lian Liu, Zeping Li, Yixing Xu, Dong Li, Lu Tian, Jie He, Ashish Sirasao, Emad Barsoum

MoEC: A Memory-Routed Mixture-of-Experts Controller for Adaptive Minecraft Control (2026.acl-long)

Challenge:	Pre-trained language models (PLMs) are robust in contextual understanding but their considerable size incurs significant computational and storage costs.
Approach:	They propose a Sparse-Dense-Sparse pruning framework to prune PLMs . they prune less critical connections using conventional pruning methods .
Outcome:	The proposed pruning framework outperforms SparseGPT and Wanda under identical sparsity.

Copied to clipboard

Hui Wu, Chao Xu, Jianghui Wang, Ziqiong Liu, Dong Li, Yiwei Dai, Emad Barsoum

Vision-aided Unsupervised Constituency Parsing with Multi-MLLM Debating (2025.findings-acl)

Challenge:	Existing systems rely on a monolithic policy to execute subgoals across varying contexts, causing inconsistent outcomes and scaling only partially mitigates.
Approach:	They propose a memory-routed mixtureof-experts controller for Adaptive Minecraft Control that routes via a subgoal-indexed expert memory and regulates capacity through failure-triggered expert growth and redundancy-aware consolidation.
Outcome:	The proposed controller shows significant gains in adaptability, robustness, and execution consistency over strong baselines.

Copied to clipboard

Dong Zhang, Haiyan Tian, Qingying Sun, Shoushan Li

Bio-RFX: Refining Biomedical Extraction via Advanced Relation Classification and Structural Constraints (2024.emnlp-main)

Challenge:	Existing approaches require explicit cross-modal alignment, but new approaches address these challenges.
Approach:	They propose a framework for vision-aided unsupervised constituency parsing . they leverage multimodal large language models pre-trained on diverse image-text or video-text data .
Outcome:	The proposed framework achieves state-of-the-art performance on image-text and video-text datasets, improving robustness and accuracy.

Copied to clipboard

Minjia Wang, Fangzhou Liu, Xiuxing Li, Bowen Dong, Zhenyu Li, Tengyu Pan, Jianyong Wang

Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models (2024.emnlp-main)

Challenge:	Existing methods for extracting structured data from unstructured texts neglect unique features of the biomedical literature, such as ambiguous entities and nested proper nouns.
Approach:	They propose a model that leverages sentence-level relation classification before entity extraction to tackle entity ambiguity.
Outcome:	The proposed model outperforms baselines in both NER and RE tasks and has competitive performance compared to the state-of-the-art fine-tuned baselines for RE.

Copied to clipboard

Wenhao Yu, Hongming Zhang, Xiaoman Pan, Peixin Cao, Kaixin Ma, Jian Li, Hongwei Wang, Dong Yu

RL-PLUS: Countering Capability Boundary Collapse of LLMs in Reinforcement Learning with Hybrid-policy Optimization (2026.acl-long)

Challenge:	Standard RALMs often neglect their intrinsic knowledge due to the interference from retrieved information.
Approach:	They propose a new approach to improve robustness of RALMs by generating sequential reading notes for each retrieved document.
Outcome:	The proposed approach outperforms standard RALMs on four open-domain QA benchmarks.

Copied to clipboard

Yihong Dong, Xue Jiang, Yongding Tao, Huanyu Liu, Kechi Zhang, Lili Mou, Rongyu Cao, Yingwei MA, Jue Chen, Binhua Li, Zhi Jin, Fei Huang, Yongbin Li, Ge Li

STEntConv: Predicting Disagreement between Reddit Users with Stance Detection and a Signed Graph Convolutional Network (2024.lrec-main)

Challenge:	Reinforcement Learning with Verifiable Reward (RLVR) has significantly advanced the complex reasoning abilities of Large Language Models (LLMs).
Approach:	They propose a hybrid-policy optimization approach that synergizes internal exploitation with external data to achieve stronger reasoning capabilities.
Outcome:	The proposed approach achieves state-of-the-art performance on six math reasoning benchmarks and superior performance on out-of distribution reasoning tasks.

Copied to clipboard

Isabelle Lorge, Li Zhang, Xiaowen Dong, Janet Pierrehumbert

SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models (2024.findings-acl)

Challenge:	Existing methods to detect disagreements on social media platforms have focused on supplementing textual information with user network information, such as Twitter's following system, retweets and hashtags.
Approach:	They propose a method which builds a graph of users and named entities and trains a Signed Graph Convolutional Network to detect disagreement between comment and reply posts.
Outcome:	The proposed model builds a graph of users and named entities weighted by stance and trains a Signed Graph Convolutional Network (SGCN) to detect disagreement between comment and reply posts.

Copied to clipboard

Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, Jing Shao

Can Language Models Understand Physical Concepts? (2023.emnlp-main)

Challenge:	SALAD-Bench is a safety benchmark specifically designed for LLMs . it provides a robust source for evaluating both attack and defense algorithms .
Approach:	They propose a hierarchical safety benchmark specifically designed for LLMs . it uses a taxonomy of questions spanning three levels and a robust taxonomies based on a QA pair .
Outcome:	The proposed safety benchmark shows that LLMs are resilient against emerging threats and the effectiveness of contemporary defense methods.

Copied to clipboard

Lei Li, Jingjing Xu, Qingxiu Dong, Ce Zheng, Xu Sun, Lingpeng Kong, Qi Liu

A.S.E: A Repository-Level Benchmark for Evaluating Security in AI-Generated Code (2026.findings-acl)

Challenge:	Existing language models do not understand basic physical concepts in the human world.
Approach:	They propose a method to transfer embodied knowledge from visual models to LMs . they use visual concepts and embodies concepts learned from interaction with the world .
Outcome:	The proposed method achieves comparable performance with scaling up parameters of LMs 134.

Copied to clipboard

Keke Lian, Wang Bin, Lei Zhang, Libo Chen, Junjie Wang, Ziming Zhao, Yujiu Yang, Miaoqian Lin, Haotong Duan, Haoran Zhao, Shuang Liao, Mingda Guo, Quan Jiazheng, Yilu Zhong, Chenhao He, Chen Zichuan, Jie Wu, Haoling Li, Zhaoxuan Li, Jiongchi Yu, Hui LI, Dong Zhang

Generative Zero-Shot Prompt Learning for Cross-Domain Slot Filling with Inverse Prompting (2023.findings-acl)

Challenge:	Existing security evaluation benchmarks lack relevance to real-world AI programming tasks . current LLMs struggle with secure coding, research shows .
Approach:	They propose a repository-level evaluation benchmark to assess security of AI-generated code.
Outcome:	The proposed framework mirrors real-world AI programming tasks and offers valuable insights into the state of AI code generation.

Copied to clipboard

Xuefeng Li, Liwen Wang, Guanting Dong, Keqing He, Jinzheng Zhao, Hao Lei, Jiachi Liu, Weiran Xu

Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats (2026.acl-industry)

Challenge:	Existing approaches to slot filling only learn surface mapping of slot types between D S and D T and get poor generalization capability or robustness.
Approach:	They propose a generative zero-shot prompt learning framework for cross-domain slot filling which improves generalization and robustness than previous work.
Outcome:	The proposed framework improves generalization and robustness on unseen slots and an efficient prompt tuning strategy boosts performance.

Copied to clipboard

Pengxiang Zhao, Hui-Ling Zhen, Xing Li, Han Bao, Weizhe Lin, Zhiyuan Yang, Yu Zi Wei, Xin Wang, Mingxuan Yuan, Xianzhi Yu, Zhenhua Dong

Unified Speech-Text Pre-training for Speech Translation and Recognition (2022.acl-long)

Challenge:	Low-bit floating-point formats like MXFP and NVFP4 offer new opportunities for precision and efficiency.
Approach:	They evaluate HiFloat (HiF8 and HiF4), a family of floating-point formats tailored for Ascend NPUs.
Outcome:	The proposed formats excel with high-variance data and are compatible with state-of-the-art quantization frameworks.

Copied to clipboard

Yun Tang, Hongyu Gong, Ning Dong, Changhan Wang, Wei-Ning Hsu, Jiatao Gu, Alexei Baevski, Xian Li, Abdelrahman Mohamed, Michael Auli, Juan Pino

MLAS-LoRA: Language-Aware Parameters Detection and LoRA-Based Knowledge Transfer for Multilingual Machine Translation (2025.acl-long)

Challenge:	Existing methods to pre-train speech and text use unlabeled data to learn universal feature representations.
Approach:	They propose a method to jointly pre-train speech and text in an encoder-decoder modeling framework for speech translation and recognition.
Outcome:	The proposed method achieves between 1.7 and 2.3 BLEU improvement above the state of the art on the MuST-C speech translation dataset and comparable WERs to wav2vec 2.0 on the Librispeech speech recognition task.

Copied to clipboard

Tianyu Dong, Bo Li, Jinsong Liu, Shaolin Zhu, Deyi Xiong

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training (2021.naacl-main)

Challenge:	Large language models (LLMs) have demonstrated strong performance even with limited parallel data.
Approach:	They propose a multiple language-aware LoRA knowledge transfer framework that selectively adapts LLMs to MT by transferring knowledge from a large teacher to a small student model.
Outcome:	The proposed framework outperforms baseline models on multilingual language pairs by +1.7 BLEU on average.

Copied to clipboard

Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou