Papers by Wei Dong

Harvesting and Refining Question-Answer Pairs for Unsupervised QA (2020.acl-main)

Challenge:	Existing benchmarks focus more on end-to-end performance, but neglect the underlying principles of knowledge acquisition and generalization.
Approach:	They propose a benchmark specifically designed to explore the problem-solving principles by decomposing 6.5K visual math problems into 10.9K step-level questions for evaluation.
Outcome:	The proposed benchmark covers 6.5K visual math problems and 10.9K step-level questions spanning 5 layers of knowledge granularity and 67 hierarchical knowledge concepts.

Copied to clipboard

Zhongli Li, Wenhui Wang, Li Dong, Furu Wei, Ke Xu

Low-Resource NLP

Challenge:	Recent research attempts to extend unsupervised question answering to settings with few or no labeled data available.
Approach:	They propose two approaches to improve unsupervised question answering . first, they harvest lexically and syntactically divergent Wikipedia questions to automatically construct a corpus of question-answer pairs . second, they take advantage of the QA model to extract more appropriate answers .
Outcome:	The proposed approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models.

Language Models as Inductive Reasoners (2024.eacl-long)

Copied to clipboard

Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, Furu Wei

UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model (2025.findings-naacl)

Challenge:	Inductive reasoning is a core component of human intelligence.
Approach:	They propose a task to induce natural language rules from natural language facts using natural language as representation for knowledge instead of formal language.
Outcome:	The proposed task surpasses baselines in both automatic and human evaluations.

Copied to clipboard

Zhaowei Li, Wei Wang, YiQing Cai, Qi Xu, Pengyu Wang, Dong Zhang, Hang Song, Botian Jiang, Zhida Huang, Tao Wang

Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalization (2026.acl-long)

Challenge:	Representative models like LLaVA and MiniGPT-4 have great capabilities in various tasks.
Approach:	They propose a unified model to represent various multi-modal tasks using a single representation.
Outcome:	The proposed model outperforms existing models in a variety of tasks while maintaining generality and scalability.

Copied to clipboard

Tian Xueyun, MingHua Ma, Bingbing Xu, Nuoyan Lyu, Wei Li, Heng Dong, Zheng Chu, Yuanzhuo Wang, Huawei Shen

CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation (2022.findings-emnlp)

Challenge:	Recent studies show that supervised fine-tuning (SFT) is a common approach for reasoning in large language models.
Approach:	They propose to use supervised fine-tuning (SFT) on chain-of-thought trajectories demonstrations . they find that incorporating negative traxories yields substantial OOD generalization gains .
Outcome:	The proposed scheme yields 5.51% OOD gain over positive-only training.

Copied to clipboard

Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei

XLM-E: Cross-lingual Language Model Pre-training via ELECTRA (2022.acl-long)

Challenge:	Named entity recognition (NER) suffers from the scarcity of annotated training data, especially for low-resource languages without labeled data.
Approach:	They propose a cross-lingual entity projection framework to enable zero-shot cross-linguistic NER with the help of a multilingual labeled sequence translation model.
Outcome:	The proposed method outperforms the baseline method on two benchmarks by a large margin of +3 7 F1 scores and achieves state-of-the-art performance.

Copied to clipboard

Zewen Chi, Shaohan Huang, Li Dong, Shuming Ma, Bo Zheng, Saksham Singhal, Payal Bajaj, Xia Song, Xian-Ling Mao, Heyan Huang, Furu Wei

SDBench: A Survey-based Domain-specific LLM Benchmarking and Optimization Framework (2025.acl-long)

Challenge:	ELECTRA-style tasks are used to pretrain cross-lingual models for NLP tasks . masked language modeling tasks require massive computation resources, rendering such models quite expensive .
Approach:	They propose to use ELECTRA-style tasks to pre-train a cross-lingual language model . they propose to pretrain the model on multilingual and parallel corpora .
Outcome:	The proposed model outperforms baseline models on cross-lingual understanding tasks with much less computation cost.

Copied to clipboard

Cheng Guo, Hu Kai, Shuxian Liang, Yiyang Jiang, Yi Gao, Xian-Sheng Hua, Wei Dong

Zero-Shot Cross-Lingual Transfer of Neural Machine Translation with Multilingual Pretrained Encoders (2021.emnlp-main)

Challenge:	acquiring domain-specific knowledge often requires professional expert manpower.
Approach:	They propose a generic framework for generating evaluation datasets for domain-specific LLMs.
Outcome:	The proposed framework reduces the reliance on expert manpower while ensuring that the collected data is uniformly distributed.

Copied to clipboard

Guanhua Chen, Shuming Ma, Yun Chen, Li Dong, Dongdong Zhang, Jia Pan, Wenping Wang, Furu Wei

Expectation Confirmation Preference Optimization for Multi-Turn Conversational Recommendation Agent (2025.findings-acl)

Challenge:	Existing work on improving cross-lingual transferability of NMT model is under-explored.
Approach:	They propose a model that leverages a multilingual pretrained encoder to improve cross-lingual transferability.
Outcome:	The proposed model outperforms mBART and m2m-100 on a zero-shot cross-lingual transfer task.

Copied to clipboard

Xueyang Feng, Jingsen Zhang, Jiakai Tang, Wei Li, Guohao Cai, Xu Chen, Quanyu Dai, Yue Zhu, Zhenhua Dong

Dialogue Systems & Conversational Agents

Challenge:	Recent advances in Large Language Models (LLMs) have propelled the development of Conversational Recommendation Agents (CRAs).
Approach:	They propose a multi-turn preference optimization paradigm that leverages Expectation Confirmation Theory to explicitly model the evolution of user satisfaction throughout multi-turned dialogues.
Outcome:	The proposed paradigm eliminates the significant sampling overhead of existing MTPO methods while ensuring the optimization process drives meaningful improvements.

CLOMO: Counterfactual Logical Modification with Large Language Models (2024.acl-long)

Copied to clipboard

Yinya Huang, Ruixin Hong, Hongming Zhang, Wei Shao, Zhicheng Yang, Dong Yu, Changshui Zhang, Xiaodan Liang, Linqi Song

ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching (2026.findings-acl)

Challenge:	Existing studies on evaluating model reasoning are limited in both form and content.
Approach:	They propose a task to cultivate counterfactual thought processes within large language models and an evaluation metric to evaluate their natural language output instead of modeling the task as a multiple-choice problem.
Outcome:	The proposed evaluation metric aligns well with human preference.

Copied to clipboard

Han Zhu, Wei Kang, Liyong Guo, Zengwei Yao, Fangjun Kuang, Weiji Zhuang, Zhaoqing Li, Zhifeng Han, Dong Zhang, Xin Zhang, Xingchen Song, Lingxuan Ye, Long Lin, Daniel Povey

Dialogue Systems & Conversational Agents

Challenge:	Existing autoregressive models for dialogue generation suffer from high latency and stability issues.
Approach:	They propose a non-autoregressive (NAR) zero-shot spoken dialogue generation model based on flow-matching.
Outcome:	The proposed model outperforms existing models in speech generation due to poor speech intelligibility and turn-taking precision.

Cooperative or Competitive? Understanding the Interaction between Attention Heads From A Game Theory Perspective (2025.acl-long)

Copied to clipboard

Xiaoye Qu, Zengqi Yu, Dongrui Liu, Wei Wei, Daizong Liu, Jianfeng Dong, Yu Cheng

Linguistics & Cognitive NLP

Challenge:	a number of attention-based large language models (LLMs) focus on individual head contributions, but the precise interaction mechanisms between attention heads remain poorly understood.
Approach:	They propose a game-theoretic attention calibration method that uses the Harsanyi dividend . they selectively retain heads demonstrating significant cooperative gains and apply fine-grained adjustments to remaining heads .
Outcome:	The proposed framework is based on the Harsanyi dividend, a concept from cooperative game theory.

Allocating Large Vocabulary Capacity for Cross-Lingual Language Model Pre-Training (2021.emnlp-main)

Copied to clipboard

Bo Zheng, Li Dong, Shaohan Huang, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

Can Monolingual Pretrained Models Help Cross-Lingual Classification? (2020.aacl-main)

Challenge:	Existing models require a more expressive vocabulary to represent all languages . however, increasing the vocabulary size significantly slows down the pre-training speed .
Approach:	They propose an algorithm VoCap to determine the desired vocabulary capacity of each language.
Outcome:	The proposed algorithm improves cross-lingual model pre-training while reducing side effects of increasing vocabulary size.

Copied to clipboard

Zewen Chi, Li Dong, Furu Wei, Xianling Mao, Heyan Huang

Distributed LLM Serving on Consumer-Grade GPUs by Reconciling Computation and Communication (2025.findings-emnlp)

Challenge:	Multilingual pretrained language models have shown impressive results for cross-lingual transfer, but due to the constant model capacity, multilingual pre-training usually lags behind the monolingual competitors.
Approach:	They propose to transfer the knowledge from monolingual pretrained models to multilingual ones to improve zero-shot cross-lingual classification by using machine translation systems.
Outcome:	The proposed methods outperform vanilla multilingual fine-tuning on two cross-lingual classification benchmarks.

Copied to clipboard

Lewei Jin, Kui Zhang, Yongqi Chen, null Zhuoyifan, Renjie Li, Yi Gao, Bowei Yang, Zhengong Cai, Wei Dong

Green & Sustainable NLP

Challenge:	Large language models are reshaping internet services, and serving them is costly.
Approach:	They propose an efficient distributed LLM serving system that splits prefill and decode requests into smaller chunks .
Outcome:	The proposed system reduces TTFT, TPOT, and latency compared to the state-of-the-art system.

Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs (2026.findings-acl)

Copied to clipboard

Jinbo Liu, Defu Cao, Yifei Wei, Tianyao Su, Yuan Liang, Yushun Dong, Yan Liu, Yue Zhao, Xiyang Hu

MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)

Challenge:	Multi-agent LLMs are rapidly moving from prototype to real-world use . network topology is a first-order security parameter in multi-aggent systems .
Approach:	They propose a framework for comparing topology-conditioned memory leakage in multi-agent LLM systems.
Outcome:	The proposed framework evaluates topology-conditioned memory leakage in multi-agent LLM systems.

Copied to clipboard

Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He

Green & Sustainable NLP

Challenge:	Document images are characterized by higher resolutions, denser content, and more complex structural layouts.
Approach:	They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition.
Outcome:	The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead.

Uncertainty-Aware Test-Time Search for Optimization Problem Solving (2026.acl-long)

Copied to clipboard

Linlin Yu, Xujiang Zhao, Dong Li, Yanchi Liu, Wei Cheng, Zhengzhang Chen, Chen Zhao, Feng Chen, Haifeng Chen

ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning (2025.emnlp-main)

Challenge:	Language model hallucinations and limited availability of labeled datasets often result in misaligned formulations, code errors and feasibility failures.
Approach:	They propose a Monte Carlo Tree Search framework that automates optimization problems from natural language descriptions with efficiency and reliability.
Outcome:	The proposed framework achieves state-of-the-art solution accuracy and reduces token usage.

Copied to clipboard

Ziqing Qiao, Yongheng Deng, Jiali Zeng, Dong Wang, Lai Wei, Guanbo Wang, Fandong Meng, Jie Zhou, Ju Ren, Yaoxue Zhang

Multimodal Dual-Path Decoding for Medical Report Generation (2026.findings-acl)

Challenge:	Existing methods for fine-tuning-based compression suffer from verbose outputs, increasing computational overhead.
Approach:	They propose a framework to generate concise reasoning chains using Confidence Injection and Early Stopping.
Outcome:	The proposed framework reduces the length of the model by up to 50% while maintaining high task accuracy.

Copied to clipboard

Jinghan Sun, Dong Wei, Zhihong Zhu, Yuyang Xue, Steven McDonagh, Xian Wu

mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs (2021.emnlp-main)

Challenge:	Current methods for radiology report generation rely on encoder-decoder based frameworks that fail to integrate multimodal clinical evidence with domain-specific knowledge.
Approach:	They propose a multimodal dual-path framework that synergistically integrates large vision-language models and large language models for radiology report generation.
Outcome:	The proposed framework improves on the public MIMIC-CXR benchmark and shows that it is superior to state-of-the-art models.

Copied to clipboard

Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang, Saksham Singhal, Xian-Ling Mao, Heyan Huang, Xia Song, Furu Wei

Mitigating the Alignment Tax of RLHF (2024.emnlp-main)

Challenge:	Multilingual T5 pretrains a sequence-to-sequence model on monolingual texts, but it has shown promising results on many cross-lingual tasks.
Approach:	They propose a partially non-autoregressive objective for text-to-text pre-training and propose mT6 to improve cross-lingual transferability over multilingual T5.
Outcome:	The proposed model improves cross-lingual transferability over existing models.

Copied to clipboard

Yong Lin, Hangyu Lin, Wei Xiong, Shizhe Diao, Jianmeng Liu, Jipeng Zhang, Rui Pan, Haoxiang Wang, Wenbin Hu, Hanning Zhang, Hanze Dong, Renjie Pi, Han Zhao, Nan Jiang, Heng Ji, Yuan Yao, Tong Zhang

“I’ve Decided to Leak”: Probing Internals Behind Prompt Leakage Intents (2025.emnlp-main)

Challenge:	Large Language Models (LLMs) acquire a wide range of abilities during pre-training, but aligning LLMs under Reinforcement Learning with Human Feedback (RLHF) can lead to forgetting pretrained abilities, which is also known as the alignment tax.
Approach:	They propose to use a model averaging technique to find the most powerful alignment-forging Pareto front among RLHF algorithms.
Outcome:	The proposed method achieves the strongest alignment-forging Pareto front among competing methods.

Copied to clipboard

Jianshuo Dong, Yutong Zhang, Liu Yan, Zhenyu Zhong, Tao Wei, Ke Xu, Minlie Huang, Chao Zhang, Han Qiu

Learning to Ask Unanswerable Questions for Machine Reading Comprehension (P19-1)

Challenge:	Large language models (LLMs) exhibit prompt leakage vulnerabilities, raising intellectual property and confidentiality concerns.
Approach:	They use probing techniques to capture LLMs’ intent-related internal representations and show that they internalize prompt leakage intents in their hidden states before generating tokens.
Outcome:	The proposed probes achieve 90%+ AUROC across all tested models, even when applied to new system prompts and attacks.

Copied to clipboard

Haichao Zhu, Li Dong, Furu Wei, Wenhui Wang, Bing Qin, Ting Liu

Low-Resource NLP

Challenge:	Existing models for extractive reading comprehension are not good at deciding whether no answer is presented in the context.
Approach:	They propose a data augmentation technique by automatically generating relevant unanswerable questions according to an answerable question paired with its corresponding paragraph that contains the answer.
Outcome:	The proposed model performs better on the SQuAD 2.0 dataset than the baseline model and the BERT-large model.

FAITH: Factuality Alignment through Integrating Trustworthiness and Honestness (2026.findings-acl)

Copied to clipboard

Xiaoning Dong, Chengyan Wu, Yajie Wen, Yu Chen, Yun Xue, Zhang Jing, Wei Xu, Bolei Ma

Information Retrieval

Challenge:	Existing approaches to correct factually inaccurate outputs are lacking the semantic richness needed to properly understand its internal states of trustworthiness and honesty.
Approach:	They propose a framework for factuality alignment that integrates natural-language uncertainty signals with external knowledge and computes confidence scores and semantic entropy from LLM outputs.
Outcome:	Extensive experiments on four knowledge-intensive benchmarks show that FAITH improves the factual accuracy and truthfulness of Large Language Models (LLMs).

CLIP Models are Few-Shot Learners: Empirical Studies on VQA and Visual Entailment (2022.acl-long)

Copied to clipboard

Haoyu Song, Li Dong, Weinan Zhang, Ting Liu, Furu Wei

Visual Data in NLP

Challenge:	Previously, CLIP was only regarded as a powerful visual encoder.
Approach:	They propose a parameter-efficient fine-tuning strategy to boost CLIP's few-shot performance on a visual entailment task without introducing any additional pre-training procedure.
Outcome:	The proposed strategy achieves competitive zero/few-shot results on visual question answering and visual entailment tasks without introducing any additional pre-training procedure.

GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator (2023.acl-long)

Copied to clipboard

Jian Yang, Shuming Ma, Li Dong, Shaohan Huang, Haoyang Huang, Yuwei Yin, Dongdong Zhang, Liqun Yang, Furu Wei, Zhoujun Li

Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension (D19-58)

Challenge:	Existing pre-training methods underutilize the benefits of language understanding for generation.
Approach:	They propose a GAN-style model for encoder-decoder pre-training with an auxiliary discriminator.
Outcome:	The proposed model outperforms existing pre-trained models and achieves state-of-the-art performance.

Copied to clipboard

Hangbo Bao, Li Dong, Furu Wei, Wenhui Wang, Nan Yang, Lei Cui, Songhao Piao, Ming Zhou

TabularMath: Understanding Math Reasoning over Tables with Large Language Models (2026.findings-acl)

Challenge:	Experimental results show that unified model outperforms other models that treat encoding and matching separately.
Approach:	They evaluate a unified model with Transformer layers for machine reading comprehension . they find that the model learns different modeling strategies compared with previous models .
Outcome:	The unified model outperforms models with Transformer layers on the machine reading comprehension task.

Copied to clipboard

Shi-Yu Tian, Zhi Zhou, Wei Dong, Kun-Yang Yu, Ming Yang, Zi-Jian Cheng, Lan-Zhe Guo, Yu-Feng Li

Rectified Sparse Attention for Efficient Long-Sequence Generation (2026.findings-acl)

Challenge:	Mathematical reasoning has long been a key benchmark for evaluating large language models.
Approach:	They propose a framework that transforms math word problems into scalable tabular reasoning tasks.
Outcome:	The proposed framework transforms math word problems into scalable and verified tabular reasoning tasks.

Copied to clipboard

Yutao Sun, Tianzhu Ye, Li Dong, Yuqing Xia, Jian Chen, Yizhao Gao, Shijie Cao, Jianyong Wang, Furu Wei

Optimizing Retrieval-Augmented Generation for E-Commerce How-To Assistance (2026.acl-industry)

Challenge:	Recent sparse decoding methods improve efficiency but suffer from KV cache misalignment, resulting in performance degradation.
Approach:	They propose a method that combines block-sparse attention with periodic dense rectification to bound error accumulation and preserve alignment with the pretraining distribution.
Outcome:	Experiments on math reasoning, language modeling, and retrieval tasks show that ReSA achieves near-lossless generation quality with significantly improved efficiency.

Copied to clipboard

Gilad Fuchs, Leonid Ekimov, Fei Dong, Jiahong Xie, Wei Liu, Maxim Manco, Alexander Nus

Information Retrieval

Challenge:	Recent advances in large language models (LLMs) have significantly improved Retrieval-Augmented Generation (RAG), enabling assistants that can reliably ground responses in external knowledge sources while maintaining high-quality natural language interaction.
Approach:	They propose a RAG-based How-To Assistant that groundes responses in a proprietary knowledge base to provide personalized customer support.
Outcome:	The proposed assistant can ground responses in a proprietary knowledge base while maintaining high-quality natural language interaction.

Knowledge Neurons in Pretrained Transformers (2022.acl-long)

Copied to clipboard

Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, Furu Wei

TermDiffuSum: A Term-guided Diffusion Model for Extractive Summarization of Legal Documents (2025.coling-main)

Challenge:	Existing studies show that pretrained language models are good at recalling factual knowledge without fine-tuning.
Approach:	They propose a method to identify neurons that express factual knowledge in pretrained Transformers by filling-in-the-blank cloze queries.
Outcome:	The proposed method can be used to edit, erase, and update factual knowledge without fine-tuning.

Copied to clipboard

Xiangyun Dong, Wei Li, Yuquan Le, Zhangyue Jiang, Junxi Zhong, Zhong Wang

CoIR: A Comprehensive Benchmark for Code Information Retrieval Models (2025.acl-long)

Challenge:	Recent studies have explored diffusion models for extractive summarization task, showcasing their remarkable capabilities.
Approach:	They propose a term-guided diffusion model for extractive summarization of legal documents that incorporates legal terminology into the model via a well-designed multifactor fusion noise weighting schedule.
Outcome:	The proposed model outperforms existing models on a self-constructed legal summarization dataset and achieves improvements of 3.10, 2.84, and 2.89 on three public datasets.

Copied to clipboard

Xiangyang Li, Kuicai Dong, Yi Quan Lee, Wei Xia, Hao Zhang, Xinyi Dai, Yasheng Wang, Ruiming Tang

Information Retrieval

Challenge:	Existing methods and benchmarks for information retrieval are inadequately representing the diversity of code in various domains and tasks.
Approach:	They propose a benchmark specifically designed to assess code retrieval capabilities.
Outcome:	The proposed benchmark aims to invigorate research in the code retrieval domain . it shares the same data schema as other popular benchmarks like MTEB and BEIR .

GAPO: Robust Advantage Estimation for Real-World Code LLMs (2026.findings-acl)

Copied to clipboard

Jianqing Zhang, Zhezheng Hao, Wei Xia, Hande Dong, Hong Wang, Chenxing Wei, Yuyan Zhou, Yubin Qi, Qiang Lin, Jian Cao

Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner (2022.findings-naacl)

Challenge:	Reinforcement learning (RL) is widely used for post-training large language models (LLMs) in code editing, but in real-world code editing scenarios, reward distributions are often skewed with unpredictable noise, leading to distorted advantage computation and increased rollout outliers.
Approach:	They propose a group-relative method that finds an interval with the highest SNR and uses the median of that interval as an adaptive Q to replace the group mean in advantage calculation.
Outcome:	The proposed method improves on nine instruction-tuned LLMs while remaining plug-and-play and efficient.

Copied to clipboard

Danilo Neves Ribeiro, Shen Wang, Xiaofei Ma, Rui Dong, Xiaokai Wei, Henghui Zhu, Xinchi Chen, Peng Xu, Zhiheng Huang, Andrew Arnold, Dan Roth

Question Answering

Challenge:	Large language models have achieved high performance on various natural language benchmarks, but the explainability of their output remains elusive.
Approach:	They propose an architecture called iterative retrieval-generation reasoner that generates an entailment tree that explains a given hypothesis by using premises from C.
Outcome:	The proposed model outperforms existing benchmarks on premise retrieval and entailment tree generation with around 300% gain in overall correctness.

Threshold Filtering Packing for Supervised Fine-Tuning: Training Related Samples within Packs (2025.naacl-long)

Copied to clipboard

Jiancheng Dong, Lei Jiang, Wei Jin, Lu Cheng

Memory-Efficient Differentiable Transformer Architecture Search (2021.findings-acl)

Challenge:	Randomly concatenating data points can lead to cross-contamination due to the significant difference in their subject matter.
Approach:	They propose a method that randomly concatenates data of varying lengths until reaching the designed maximum length to optimize context length and reduce padding.
Outcome:	The proposed method significantly improves performance on GSM8K and HumanEval, and also improves fairness and accuracy by 15%.

Copied to clipboard

Yuekai Zhao, Li Dong, Yelong Shen, Zhihua Zhang, Furu Wei, Weizhu Chen

PhysPRM: A Generative Process Reward Model with Fine-grained Diagnosis for Physics Problem Solving (2026.findings-acl)

Challenge:	Current neural architecture search methods suffer from huge computational cost.
Approach:	They propose a reversible recursive backpropagation algorithm that uses the last layer to store the outputs of the network.
Outcome:	The proposed algorithm outperforms standard Transformers on three sequence-to-sequence datasets.

Copied to clipboard

Yuxuan Dong, Xinyu Zhang, Lingling Zhang, Han Lai, Pengyu Li, Bifan Wei, Yaqiang Wu, Jun Liu

Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs (2026.acl-long)

Challenge:	Existing Large Language Models (LLMs) struggle with physics problem solving due to difficulties in decoding implicit constraints and maintaining physical consistency.
Approach:	They propose a Generative PRM that treats evaluation as a generative task . it produces fine-grained diagnoses comprising critiques, final judgments, and specific error types .
Outcome:	The proposed model improves performance across seven benchmarks in Best-of-N and critique refinement strategies.

Copied to clipboard

Tingchao Fu, Wenkai Wang, Fanxiao Li, Huadong Zhang, Jinhong Zhang, Dayang Li, Yunyun Dong, Renyang Liu, Wei Zhou

LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models (2024.naacl-demo)

Challenge:	Existing knowledge editing paradigms suffer from editing decoupling failures . entity knowledge is sequestered into disentangled modality-specific pathways .
Approach:	They propose a method that explicitly disentangles and localizes modality-specific neuron groups for targeted knowledge.
Outcome:	The proposed method outperforms baselines in reliability and consistency while preserving model locality.

Copied to clipboard

Shizhe Diao, Rui Pan, Hanze Dong, KaShun Shum, Jipeng Zhang, Wei Xiong, Tong Zhang

Introducing Semantics into Speech Encoders (2023.acl-long)

Challenge:	Foundation models have demonstrated a great ability to achieve general human-level intelligence far beyond traditional approaches.
Approach:	They propose a toolkit to simplify the finetuning of general foundation models.
Outcome:	The proposed toolkit simplifies the domain- and task-aware finetuning of general foundation models with limited computing resources.

Copied to clipboard

Derek Xu, Shuyan Dong, Changhan Wang, Suyoun Kim, Zhaojiang Lin, Bing Liu, Akshat Shrivastava, Shang-Wen Li, Liang-Hsuan Tseng, Guan-Ting Lin, Alexei Baevski, Hung-yi Lee, Yizhou Sun, Wei Wang

Calibrating Inference Time Alignment with Sequence-level Risk Accumulation (2026.acl-long)

Challenge:	Existing self-supervised speech encoders contain primarily acoustic rather than semantic information.
Approach:	They propose a task-agnostic unsupervised way to incorporate semantic information from large language model (LLM) systems into self-supervised speech encoders without labeled audio transcriptions.
Outcome:	The proposed approach improves spoken language understanding (SLU) performance by over 5% on intent classification (IC), with modest gains in named entity resolution (NER) and slot filling (SF), and spoken question answering (SQA) score by over 22%.

Copied to clipboard

Shanwen Tan, Ziyang Dong, Wei Ju, Yiwei Fu, Hao Wu, Kun Wang, Yifan Wang, Ziyue Qiao

Mitigating Over-Refusal in Aligned Large Language Models via Inference-Time Activation Energy (2026.acl-long)

Challenge:	Existing approaches to decode large language models (LLMs) often over-reject benign information, limiting their generalizability in real-world scenarios where harmful and benign information coexist.
Approach:	They propose a framework to regulate decoding alignments for Large Language Models (LLMs) they employ a reward-guided branch decoding paradigm to incorporate safety awareness during generation.
Outcome:	The proposed framework achieves superior performance on four attack benchmarks and two neutral datasets.

Copied to clipboard

Eric Hanchen Jiang, Weixuan Ou, Run Liu, Shengyuan Pang, Guancheng Wan, Ranjie Duan, Wei Dong, Kai-Wei Chang, XiaoFeng Wang, Ying Nian Wu, Xinfeng Li

Controllable Natural Language Generation with Contrastive Prefixes (2022.findings-acl)

Challenge:	Existing safety alignment techniques prioritize mitigating harmful responses at the expense of overcautious behavior, leading models to incorrectly refuse benign requests.
Approach:	They propose a fine-tuning free framework to improve safety and reduce false refusals by dynamic, inference-time intervention.
Outcome:	The proposed framework raises compliance on the ORB-H benchmark from 57.3% to 82.6% while maintaining the baseline safety performance.

Copied to clipboard

Jing Qian, Li Dong, Yelong Shen, Furu Wei, Weizhu Chen

MARIO-0.5B: A Multi-Agent Lightweight Model for Real-Time Open Information Extraction in Low-Resource Settings (2025.findings-emnlp)

Challenge:	Existing work on controllable natural language generation has focused on fine-tuning existing models or using attribute discriminators.
Approach:	They propose a lightweight framework for controllable GPT2 generation that utilizes attribute-specific vectors to steer natural language generation.
Outcome:	The proposed framework can guide generation towards desired attributes while keeping high linguistic quality.

Copied to clipboard

Donghai Zhang, SHuangtao Yang, Dong Xiaozheng, Wei Song, Bo Fu

Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)

Challenge:	Large language models have shown remarkable capabilities in open information extraction, but their resource requirements often restrict their deployment in resource-constrained industrial settings.
Approach:	They introduce an ultra-lightweight large language model trained on instruction-based samples in Chinese, English, Korean, and Russian.
Outcome:	The proposed model outperforms large-scale models with up to 70B parameters, reducing computational resources by 140x and delivering 11x faster response times.

Copied to clipboard

Binghai Wang, Rui Zheng, Lu Chen, Zhiheng Xi, Wei Shen, Yuhao Zhou, Dong Yan, Tao Gui, Qi Zhang, Xuanjing Huang

Challenge:	Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values.
Approach:	They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values .
Outcome:	The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets.

Premise-based Multimodal Reasoning: Conditional Inference on Joint Textual and Visual Clues (2022.acl-long)

Copied to clipboard

Qingxiu Dong, Ziwei Qin, Heming Xia, Tian Feng, Shoujie Tong, Haoran Meng, Lin Xu, Zhongyu Wei, Weidong Zhan, Baobao Chang, Sujian Li, Tianyu Liu, Zhifang Sui

THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption (2022.findings-acl)

Challenge:	Existing work in vision language cross-modal reasoning uses binary or multi-choice classification based on source image and textual query.
Approach:	They propose a task where a textual premise is the background presumption on each source image.
Outcome:	The proposed task is based on a dataset of 15,360 movie screenshots and human-curated premise templates from 6 pre-defined categories.

Copied to clipboard

Tianyu Chen, Hangbo Bao, Shaohan Huang, Li Dong, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, Furu Wei

Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoning (2026.findings-acl)

Challenge:	enabling pre-trained models inference on ciphertext data is difficult due to the complex computations in transformer blocks.
Approach:	They propose an approximation approach for transformers which enables inference on ciphertext data.
Outcome:	The proposed approach can infer pre-trained models on encrypted data with negligible performance drop but enjoy theory-guaranteed privacy-preserving advantage.

Copied to clipboard

Haolin Liu, Dian Yu, Sidi Lu, Yujun Zhou, Rui Liu, Zhenwen Liang, Haitao Mi, Chen-Yu Wei, Dong Yu

AGTAO: Robust and Stabilized LLM Unlearning via Adversarial Gating Training with Adaptive Orthogonality (2026.findings-acl)

Challenge:	Existing reinforcement learning methods rely on sparse outcome rewards, which fail to credit correct intermediate steps in partially successful solutions.
Approach:	They propose a process reward model that rewards correct steps only when they detect errors . they propose VPPO, which rewards the correct prefix and an erroneous suffix .
Outcome:	a new approach outperforms sparse-reward RL and prior PRM-guided baselines on Pass@1 and Pass@K . a process reward model (PRM) outperformed sparser-rebound RL on multiple reasoning benchmarks .

Copied to clipboard

Pengyu Li, Lingling Zhang, Zhitao Gao, Yanrui Wu, Yuxuan Dong, Huan Liu, Bifan Wei, Jun Liu

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023.findings-acl)

Challenge:	Large Language Models (LLMs) unintentionally memorize sensitive data, posing privacy and security risks.
Approach:	They propose a framework that reconciles unlearning efficacy and utility preservation by using a latent-space gating mechanism to simulate internal recovery attempts.
Outcome:	The proposed framework achieves superior trade-off between unlearning efficacy and model utility.

Copied to clipboard

Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei

Interleaved Latent Visual Reasoning with Selective Perceptual Modeling (2026.acl-long)

Challenge:	Large pretrained language models have shown surprising in-context learning ability . despite the great success in performance, its working mechanism remains unclear .
Approach:	They explain language models as meta-optimizers and understand in-context learning as implicit finetuning . they find that Transformer attention has a dual form of gradient descent .
Outcome:	The proposed model can predict labels for unseen inputs without parameter updates . the proposed model outperforms smaller models with a single parameter update .

Copied to clipboard

Shuai Dong, Siyuan Wang, Xingyu Liu, Chenglin Li, Haowen Hou, Zhongyu Wei

Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning (2025.coling-main)

Challenge:	Existing approaches to interleaved reasoning are limited by the cost of re-encoding pixel-dense images.
Approach:	They propose a framework that unifies dynamic state evolution with precise perceptual modeling.
Outcome:	The proposed framework outperforms existing approaches on multimodal reasoning benchmarks.

Copied to clipboard

Xiaoye Qu, Jiashuo Sun, Wei Wei, Daizong Liu, Jianfeng Dong, Yu Cheng

StableMoE: Stable Routing Strategy for Mixture of Experts (2022.acl-long)

Challenge:	Large Vision-Language Models (LVLMs) have impressive capabilities in multi-modal context comprehension, but they still suffer from hallucination problems due to inconsistent outputs with the image content.
Approach:	They propose a training-free framework MVP to reduce hallucinations in Large Vision-Language Models . they propose multi-view information-seeking strategy to perceive the comprehensive information in the image .
Outcome:	The proposed framework reduces hallucinations in large vision-language models by combining multi-view multi-path reasoning with multi-vision multi-path reasoning.

Copied to clipboard

Damai Dai, Li Dong, Shuming Ma, Bo Zheng, Zhifang Sui, Baobao Chang, Furu Wei

Challenge:	Existing learning-to-route methods suffer from the routing fluctuation issue . with the model scale growing, training speed will go slower and memory requirements are heavy .
Approach:	They propose a Mixture-of-Experts technique that can scale up the model size of Transformers with an affordable computational overhead.
Outcome:	The proposed method outperforms existing learning-to-route methods on language modeling and multilingual machine translation.

How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition (2024.acl-long)

Copied to clipboard

Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, Jingren Zhou

Investigating Learning Dynamics of BERT Fine-Tuning (2020.aacl-main)

Challenge:	supervised fine-tuning (SFT) is a technique used to enhance multiple abilities in large language models.
Approach:	They propose to study the interplay of data composition between mathematical reasoning, code generation, and general human-aligning abilities during supervised fine-tuning.
Outcome:	The proposed model improves math reasoning and code generation with increasing data amount . the proposed model size and SFT strategies can be used to learn multiple skills with different scaling patterns.

Copied to clipboard

Yaru Hao, Li Dong, Furu Wei, Ke Xu

ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Models (2024.findings-acl)

Challenge:	Recent studies have shown that the fine-tuning process improves performance on downstream tasks.
Approach:	They propose two new pre-training tasks to improve the model performance on downstream tasks.
Outcome:	The proposed model achieves state-of-the-art on a wide array of NLP tasks.

Copied to clipboard

Haoran Luo, Haihong E, Zichen Tang, Shiyao Peng, Yikai Guo, Wentai Zhang, Chenghao Ma, Guanting Dong, Meina Song, Wei Lin, Yifan Zhu, Anh Tuan Luu

DataSciBench: An LLM Agent Benchmark for Data Science (2026.findings-acl)

Challenge:	Existing KBQA methods address inefficient knowledge retrieval and semantic parsing errors.
Approach:	They propose a generatethen-retrieve KBQA framework that generates logical form and replaces entities and relations with an unsupervised retrieval method to improve both generation and retrieval more directly.
Outcome:	Experimental results show that ChatKBQA achieves new state-of-the-art performance on standard KBQA datasets, WebQSP, and CWQ.

Copied to clipboard

Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, Yisong Yue

Challenge:	Existing benchmarks focus on single task, simple evaluation metrics, and readily available ground truth (GT) DataSciBench is built on curated, natural, and challenging prompts with complex evaluation criteria and uncertain GT.
Approach:	They propose a benchmark for evaluating Large Language Models in data science that integrates LLM-based self-consistency and human verification to ensure accuracy.
Outcome:	The proposed framework outperforms open-source models in all metrics and offers rigorous insights into LLM strengths and weaknesses.

Pre-Training to Learn in Context (2023.acl-long)

Copied to clipboard

Yuxian Gu, Li Dong, Furu Wei, Minlie Huang

SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling (2025.findings-acl)

Challenge:	Pre-trained language models are not explicitly trained to learn in context.
Approach:	They propose a framework to enhance in-context learning by pre-training language models on a large collection of "intrinsic tasks" they evaluate the in-constitution learning performance of the model trained with PICL on seven widely-used text classification datasets and the Super-NaturalInstrctions benchmark .
Outcome:	The proposed framework outperforms larger language models with nearly 4x parameters on seven widely-used datasets and the Super-NaturalInstrctions benchmark.

Copied to clipboard

Haoran Wang, Zhenyu Hou, Yao Wei, Jie Tang, Yuxiao Dong

Challenge:	Large language models (LLMs) have advanced rapidly from conversational problem solving to addressing real-world tasks involving tool use, such as software engineering (SWE).
Approach:	They propose to build an LLM-based software engineering agent that synthesizes test cases and scales up agent trajectories to build training data.
Outcome:	The proposed model outperforms state-of-the-art models on the SWE-bench-Verified benchmark.

SATA: A Paradigm for LLM Jailbreak via Simple Assistive Task Linkage (2025.findings-acl)

Copied to clipboard

Xiaoning Dong, Wenbo Hu, Wei Xu, Tianxing He

Plug-and-Play Data Module for Code RL: Adaptive Ambiguity Replay (2026.findings-acl)

Challenge:	Existing methods to design sophisticated instructions for the LLM to follow, or rely on multiple iterations, could hinder the performance and efficiency of jailbreaks.
Approach:	They propose a simple assistive task linkage paradigm which masks harmful keywords within malicious queries and uses a masked language model task to encode the semantics of the mangled keywords.
Outcome:	The proposed paradigm can effectively circumvent LLM safeguards and elicit harmful responses.

Copied to clipboard

Jianqing Zhang, Wei Xia, Zhezheng Hao, Hong Wang, Hande Dong, Qiang Lin, Yang Liu, Jian Cao, Qiang Yang

Towards Stable and Effective Reinforcement Learning for Mixture-of-Experts (2026.acl-long)

Challenge:	Existing approaches to reinforcement learning (RL) rely on static, in-epoch metrics that overlook training dynamics, often introducing low-utility or outdated data.
Approach:	They propose a plug-and-play module that prioritizes cross-epoch ambiguous samples to neutralize the noise from stale experiences.
Outcome:	Extensive experiments on nine LLMs show that Adaptive Ambiguity Replay outperforms state-of-the-art baselines on real-world code editing tasks.

Copied to clipboard

Di Zhang, Xun Wu, Shaohan Huang, Lingjie Jiang, Yaru Hao, Li Dong, Zewen Chi, Zhifang Sui, Furu Wei

SHIELD: LLM-Driven Schema Induction for Predictive Analytics in EV Battery Supply Chain Disruptions (2024.emnlp-industry)

Challenge:	Reinforcement learning with verifiable rewards (RLVR) training with Mixture-of-Experts policies remains fragile and prone to reward collapse.
Approach:	They propose a router shift-based policy optimization method that computes a per-token router-shift ratio conditioned on the previously activated experts and applies stop-gradient and a lower-bound floor.
Outcome:	The proposed method achieves better performance and greater stability than previous methods.

Copied to clipboard

Zhi-Qi Cheng, Yifei Dong, Aike Shi, Wei Liu, Yuzhi Hu, Jason O’Connor, Alexander Hauptmann, Kate Whitefoot

ReasMark: A Robust Watermark for Attributing LLM Reasoning Under Knowledge Distillation Attacks (2026.acl-long)

Challenge:	EV battery supply chain is vulnerable to disruptions caused by natural disasters and geopolitical tensions.
Approach:	They propose a system integrating Large Language Models with domain expertise for EV supply chain risk assessment.
Outcome:	Evaluated on 12,070 paragraphs from 365 sources (2022-2023), SHIELD outperforms baseline GCNs and LLM+prompt methods in disruption prediction.

Copied to clipboard

Peizhuo Lv, Ruihua Zhou, Yunpeng Li, Ruigang Liang, Xingshuo Han, XiaoFeng Wang, Wei Dong, Yuling Liu

Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement (2025.acl-long)

Challenge:	Existing reasoning-enhanced large language models fail to provide reliable attribution of reasoning behavior once it is transferred through knowledge distillation.
Approach:	They propose to embed a reasoning-length gap in a model by querying a target domain and training a local student to imitate its outputs.
Outcome:	et al. show that ReasMark outperforms baselines while preserving task utility.

Copied to clipboard

Yichen Dong, Xinglin Lyu, Junhui Li, Daimeng Wei, Min Zhang, Shimin Tao, Hao Yang

Machine Translation

Challenge:	Recent research has shown that large language models (LLMs) can enhance translation quality through self-refinement.
Approach:	They propose to extend translation refinement from sentence-level to document-level by using document-to-document (Doc2Doc) translations.
Outcome:	The proposed method improves translation quality across ten translation tasks with LLaMA-3-8B-Instruct and Mistral-Nemo-Instru.

Train in Vain: Functionality-Preserving Poisoning to Prevent Unauthorized Use of Code Datasets (2026.findings-acl)

Copied to clipboard

Yuan Xiao, Jiaming Wang, Yuchen Chen, Wei Song, Jun Sun, Shiqing Ma, Yanzhou Mu, Juan Zhai, Chunrong Fang, Jin Song Dong, Zhenyu Chen

Programming Languages in NLP

Challenge:	Existing methods for dataset poisoning require full-dataset poison, which breaks code compilability.
Approach:	They propose a functionality-preserving poisoning approach that injects short, compilable weak-use fragments into executed code paths.
Outcome:	The proposed method contaminates 10% of the dataset while maintaining 100% compilability and functional correctness.

Revisiting the Reliability of Language Models in Instruction-Following (2026.acl-long)

Copied to clipboard

Jianshuo Dong, Yutong Zhang, Liu Yan, Zhenyu Zhong, Tao Wei, Chao Zhang, Han Qiu

ROMA: Real-time Omni-Multimodal Assistant with Interactive Streaming Understanding (2026.findings-acl)

Challenge:	Several benchmarks have been proposed to measure instruction-following accuracy, but these scores do not translate to reliable services in real-world use.
Approach:	They propose a new metric reliable@k and develop an automated pipeline to generate cousin prompts.
Outcome:	The proposed model can be instantiated with cousin prompts and generates high-quality cousin prompt data.

Copied to clipboard

Tian Xueyun, Wei Li, Bingbing Xu, Heng Dong, Yuanzhuo Wang, Huawei Shen

Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study (2023.emnlp-main)

Challenge:	Existing omni-multimodal large language models lack incomplete modality support or lack autonomous proactive monitoring.
Approach:	They propose a real-time omni-multimodal assistant for unified reactive and proactive interaction that decouples response initiation from generation to ensure precise triggering without task conflict.
Outcome:	The proposed model achieves state-of-the-art performance on proactive tasks while competing in reactive settings.

Copied to clipboard

Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro

Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning (2023.acl-long)

Challenge:	a recent study shows that retrieval-augmented LMs can improve text generation quality and accuracy.
Approach:	They propose a model that reproduces RETRO parameters while retrieving a text corpus . they find RETRO outperforms GPT on text generation with less repetition .
Outcome:	The proposed model outperforms standard retrieval-augmented GPT and retrieval augmented GTP on text generation and accuracy tasks.

Copied to clipboard

Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song

Semi-Supervised Disfluency Detection (C18-1)

Challenge:	XY-LENT: X-Y bitext enhanced Language ENcodings achieves state-of-the-art performance over 5 cross-lingual tasks within all model size bands.
Approach:	They propose a method for building multilingual representation models that are competitive with existing models and more parameter efficient.
Outcome:	The proposed model outperforms XLM-R XXL and is 5x and 6x smaller respectively.

Copied to clipboard

Feng Wang, Wei Chen, Zhen Yang, Qianqian Dong, Shuang Xu, Bo Xu

Uncertainty Propagation on LLM Agent (2025.acl-long)

Challenge:	Detecting disfluency can be difficult because of the flexible nature of reparandum structure and the lack of a nested structure.
Approach:	They propose a semi-supervised approach which extracts hidden features from self-attention without any Recurrent Neural Network (RNN) or Convolutional Neural Net (CNN).
Outcome:	The proposed approach improves over baselines by using unlabelled data . identifying and removing non-fluent factors would help to improve spontaneous speech quality .

Copied to clipboard

Qiwei Zhao, Dong Li, Yanchi Liu, Wei Cheng, Yiyou Sun, Mika Oishi, Takao Osaki, Katsushi Matsuda, Huaxiu Yao, Chen Zhao, Haifeng Chen, Xujiang Zhao

Challenge:	Existing methods for estimating uncertainty in large language models (LLMs) focus on final-step outputs, which fail to account for cumulative uncertainty over multi-step decision-making process and dynamic interactions between agents and their environments.
Approach:	They propose a framework that propagates uncertainty through each step of an LLM-based agent’s reasoning process.
Outcome:	Extensive experiments on benchmark datasets show that the proposed framework outperforms state-of-the-art methods by 20%.

Zero-Shot Cross-Domain Aspect-Based Sentiment Analysis via Domain-Contextualized Chain-of-Thought Reasoning (2025.findings-emnlp)

Copied to clipboard

Chuming Shen, Wei Wei, Dong Wang, Zhong-Hao Wang

Low-Resource NLP

Challenge:	Cross-domain aspect-based sentiment analysis (ABSA) aims to learn specific knowledge from a source domain to perform various tasks on a target domain.
Approach:	a new framework is proposed to learn specific knowledge from a source domain . the framework uses domain adaptation techniques to transfer domain-agnostic features .
Outcome:	a new learning framework for cross-domain aspect-based sentiment analysis is proposed . it effectively eliminates dependency on target-domain annotations, authors say .

Joint Multi-Label Attention Networks for Social Text Annotation (N19-1)

Copied to clipboard

Hang Dong, Wei Wang, Kaizhu Huang, Frans Coenen

Challenge:	Present research shows that title metadata could affect social annotation.
Approach:	They propose a title-guided attention network for document annotation with user-generated tags that separates the title from the content of a document and applies a semantic-based loss regulariser over each sentence in the content.
Outcome:	The proposed approach outperforms the Bi-GRU and Hierarchical Attention Network (HAN) on two open datasets with 10%-30% reduction in training time.

Learning to Sample Replacements for ELECTRA Pre-Training (2021.findings-acl)

Copied to clipboard

Yaru Hao, Li Dong, Hangbo Bao, Ke Xu, Furu Wei

A Length-Extrapolatable Transformer (2023.acl-long)

Challenge:	Experimental results show that ELECTRA pretrains a discriminator to detect replaced tokens . despite compelling performance, there is no direct feedback loop from discriminator and generator to generator, making replacements biased to correct tokens.
Approach:	They propose to augment sampling with a hardness prediction mechanism to encourage the discriminator to learn what it has not acquired.
Outcome:	The proposed method improves ELECTRA pre-training on various downstream tasks.

Copied to clipboard

Yutao Sun, Li Dong, Barun Patra, Shuming Ma, Shaohan Huang, Alon Benhaim, Vishrav Chaudhary, Xia Song, Furu Wei

Visualizing and Understanding the Effectiveness of BERT (D19-1)

Challenge:	Existing Transformers can only deal with the in-distribution size of inputs.
Approach:	They propose a relative position embedding to explicitly maximize attention resolution . they also use blockwise causal attention during inference for better resolution a .
Outcome:	The proposed model achieves strong performance in interpolation and extrapolation settings.

Copied to clipboard

Yaru Hao, Li Dong, Furu Wei, Ke Xu

Consistency Regularization for Cross-Lingual Fine-Tuning (2021.acl-long)

Challenge:	Language model pre-training, such as BERT, has achieved strong performance in many NLP tasks.
Approach:	They propose to visualize loss landscapes and optimization trajectories of fine-tuning BERT on specific datasets.
Outcome:	The proposed model improves performance and generalization capability across tasks.

Copied to clipboard

Bo Zheng, Li Dong, Shaohan Huang, Wenhui Wang, Zewen Chi, Saksham Singhal, Wanxiang Che, Ting Liu, Xia Song, Furu Wei

GenProve: Learning to Generate Text with Fine-Grained Provenance (2026.acl-long)

Challenge:	Experimental results show that consistency regularization improves cross-lingual fine-tuning . pre-trained cross-linguistic models can transfer task-specific supervision from one language to the other .
Approach:	They propose to improve cross-lingual fine-tuning with consistency regularization . they use example consistency regularized to penalize prediction sensitivity to four types of data augmentations .
Outcome:	The proposed method improves cross-lingual fine-tuning across tasks . it can be generalized to other target languages without additional training .

Copied to clipboard

Jingxuan Wei, Xingyue Wang, Yanghaoyu Liao, Jie Dong, Yuchen Liu, Caijun Jia, Bihui Yu, Junnan Zhu

Improving Pretrained Cross-Lingual Language Models via Self-Labeled Word Alignment (2021.acl-long)

Challenge:	Existing methods for large language models (LLMs) are coarse-grained and fail to distinguish between direct quotes and complex reasoning.
Approach:	They propose a framework that combines supervised fine-tuning and group relative policy optimization to generate fluent answers while simultaneously producing sentence-level provenance triples.
Outcome:	The proposed framework outperforms 14 strong large language models in joint evaluation.

Copied to clipboard

Zewen Chi, Li Dong, Bo Zheng, Shaohan Huang, Xian-Ling Mao, Heyan Huang, Furu Wei

Event Detection from Social Media for Epidemic Prediction (2024.naacl-long)

Challenge:	Experimental results show that denoising word alignment improves cross-lingual transferability . most applications and resources are still English-centric, making non-English users hard to access.
Approach:	They propose to denoise word alignment as a cross-lingual pre-training task . they first self-label word alignments for parallel sentences and then mask tokens .
Outcome:	The proposed model improves cross-lingual transferability on token-level tasks, especially on question answering, and structured prediction.

Copied to clipboard

Tanmay Parekh, Anh Mac, Jiarui Yu, Yuxuan Dong, Syed Shahriar, Bonnie Liu, Eric Yang, Kuan-Hao Huang, Wei Wang, Nanyun Peng, Kai-Wei Chang

MiniLMv2: Multi-Head Self-Attention Relation Distillation for Compressing Pretrained Transformers (2021.findings-acl)

Challenge:	Social media is an easy-to-access platform providing timely updates about societal trends and events.
Approach:	They propose a framework to extract epidemic-related events from social media posts to provide early warnings.
Outcome:	The proposed framework can detect epidemic events for three unseen epidemics of Monkeypox, Zika, and Dengue while existing models fail miserably.

Copied to clipboard

Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, Furu Wei

Adapt-and-Distill: Developing Small, Fast and Effective Pretrained Language Models for Domains (2021.findings-acl)

Challenge:	Existing work on deep self-attention distillation for natural language processing tasks is limited by computational resources and latency.
Approach:	They generalize deep self-attention distillation in MINILM by using only self- attention relation distillation for taskagnostic compression of pretrained Transformers.
Outcome:	The proposed model outperforms the state-of-the-art in a multilingual and multilingual teacher model.

Copied to clipboard

Yunzhi Yao, Shaohan Huang, Wenhui Wang, Li Dong, Furu Wei

Breaking the Ceiling: Exploring the Potential of Jailbreak Attacks through Expanding Strategy Space (2025.findings-acl)

Challenge:	Large pre-trained models suffer from domain shift and are not optimal for specific domains.
Approach:	They propose a general approach to developing small, fast and effective pretrained models for specific domains by adapting off-the-shelf general pretrained model and performing task-agnostic knowledge distillation in target domains.
Outcome:	The proposed approach achieves better performance over the BERT BASE model in domain-specific tasks while 3.3 smaller and 5.1 faster than the BRT BASE.

Copied to clipboard

Yao Huang, Yitong Sun, Shouwei Ruan, Yichi Zhang, Yinpeng Dong, Xingxing Wei

IoTMigrator: LLM-driven Embedded IoT Code Migration across Different OSes for Cloud-device Integration (2025.findings-emnlp)

Challenge:	Existing methods to exploit black-box jailbreaks fail to capture key attack patterns . a novel framework decomposes jailbreak strategies into essential components .
Approach:	They propose a framework that decomposes jailbreak strategies into essential components and develops genetic-based optimization with intention evaluation mechanisms.
Outcome:	The proposed framework achieves 90% success rate on Claude-3.5, where prior methods completely fail . it also surpasses specialized safeguard models in evaluation accuracy .

Copied to clipboard

null Yq, Kaijie Gong, Yi Gao, Hao Wang, Wei Dong

Challenge:	Neither outline-based code generation nor common code translation techniques can adequately address this challenge, despite their prevalence in existing systems.
Approach:	They have developed an algorithm that employs a multi-agent pipeline to handle embedded code migration under the TSL paradigm.
Outcome:	The proposed algorithm outperforms the baseline by 50.5% for pass rate and 13.0% for completeness across all tasks in RIOT and Zephyr.

UnifiedVisual: A Framework for Constructing Unified Vision-Language Datasets (2025.emnlp-main)

Copied to clipboard

Pengyu Wang, Shaojun Zhou, Chenkun Tan, Xinghao Wang, Wei Huang, Zhen Ye, Zhaowei Li, Botian Jiang, Dong Zhang, Xipeng Qiu

Document Hashing with Mixture-Prior Generative Models (D19-1)

Challenge:	Existing datasets address understanding and generation in isolation, limiting the performance of unified vision large language models.
Approach:	They propose a dataset that facilitates mutual enhancement between multimodal understanding and generation.
Outcome:	The proposed framework integrates diverse visual and textual inputs and outputs, enabling comprehensive cross-modal reasoning and precise text-to-image alignment.

Copied to clipboard

Wei Dong, Qinliang Su, Dinghan Shen, Changyou Chen

Challenge:	Existing generative hashing methods only consider the use of simple priors, which limits them to further improve their performance.
Approach:	They propose to use Gaussian and Bernoulli priors to generate hashing codes . they propose to cast a Gausssian latent representation into binary code .
Outcome:	The proposed models outperform existing methods on a benchmark dataset using Gaussian and Bernoulli priors.

Continual Named Entity Recognition without Catastrophic Forgetting (2023.emnlp-main)

Copied to clipboard

Duzhen Zhang, Wei Cong, Jiahua Dong, Yahan Yu, Xiuyi Chen, Yonggang Zhang, Zhen Fang

Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data (2026.acl-long)

Challenge:	Named Entity Recognition (CNER) is a burgeoning area of research . a new paradigm has ushered NER into a non-entity type at the current step t .
Approach:	They propose a pooled feature distillation loss that skillfully navigates the trade-off between retaining knowledge of old entity types and acquiring new ones.
Outcome:	The proposed method outperforms state-of-the-art approaches on ten CNER settings using three datasets.

Copied to clipboard

FengXian Dong, Zhi Zheng, Xiao Han, Wei Chen, Jingqing Ruan, Tong Xu, Yong Chen, Enhong Chen

Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats (2026.acl-industry)

Challenge:	Existing methods for automated feature generation rely on predefined operator libraries and do not incorporate feature semantics, limiting their ability to produce high-quality features.
Approach:	They propose a Memory-Augmented LLM-based Multi-Agent System (MALMAS) that decomposes the generation process into agents with distinct responsibilities.
Outcome:	The proposed method extracts informative features from raw tabular data without manual intervention and is crucial for accurate, generalizable machine learning.

Copied to clipboard

Pengxiang Zhao, Hui-Ling Zhen, Xing Li, Han Bao, Weizhe Lin, Zhiyuan Yang, Yu Zi Wei, Xin Wang, Mingxuan Yuan, Xianzhi Yu, Zhenhua Dong

Green & Sustainable NLP

Challenge:	Low-bit floating-point formats like MXFP and NVFP4 offer new opportunities for precision and efficiency.
Approach:	They evaluate HiFloat (HiF8 and HiF4), a family of floating-point formats tailored for Ascend NPUs.
Outcome:	The proposed formats excel with high-variance data and are compatible with state-of-the-art quantization frameworks.

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training (2021.naacl-main)

Copied to clipboard

Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, Heyan Huang, Ming Zhou

NL2Formula: Generating Spreadsheet Formulas from Natural Language Queries (2024.findings-eacl)

Challenge:	Existing methods for learning cross-lingual representations are lacking in the field of NLP.
Approach:	They propose a framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.
Outcome:	The proposed approach improves cross-lingual transferability on benchmarks.

Copied to clipboard

Wei Zhao, Zhitao Hou, Siyuan Wu, Yan Gao, Haoyu Dong, Yao Wan, Hongyu Zhang, Yulei Sui, Haidong Zhang