Papers by Tao Wei
A Survey of Large Language Models in Psychotherapy: Current Landscape and Future Directions (2025.findings-acl)
Copied to clipboard
Hongbin Na, Yining Hua, Zimu Wang, Tao Shen, Beibei Yu, Lilin Wang, Wei Wang, John Torous, Ling Chen
| Challenge: | Large language models (LLMs) can handle extensive context and multi-turn reasoning. |
| Approach: | They propose a taxonomy dividing psychotherapy into stages of assessment, diagnosis, and treatment to examine LLM advancements and challenges. |
| Outcome: | The proposed taxonomy reveals imbalances in current research, such as a focus on common disorders, linguistic biases, fragmented methods, and limited theoretical integration. |
Better Process Supervision with Bi-directional Rewarding Signals (2025.findings-acl)
Copied to clipboard
Wenxiang Chen, Wei He, Zhiheng Xi, Honglin Guo, Boyang Hong, Jiazheng Zhang, Nijun Li, Tao Gui, Yun Li, Qi Zhang, Xuanjing Huang
| Challenge: | Existing processes that reward for each step are one-directional and lack a mechanism to model the distance to the final target. |
| Approach: | They propose a process supervision model that evaluates the correctness of previous steps and the probability of future success. |
| Outcome: | The proposed model outperforms existing supervision models like ORM and PRM on reasoning tasks and improves solution re-design. |
RE-Matching: A Fine-Grained Semantic Matching Method for Zero-Shot Relation Extraction (2023.acl-long)
Copied to clipboard
Jun Zhao, WenYu Zhan, Xin Zhao, Qi Zhang, Tao Gui, Zhongyu Wei, Junzhe Wang, Minlong Peng, Mingming Sun
| Challenge: | Existing methods for zero-shot relation extraction lack explicit modeling of matching pattern . et al. (2018) show that our method achieves higher matching accuracy and faster inference speed . |
| Approach: | They propose a fine-grained semantic matching method tailored for zero-shot relation extraction . they decompose sentence-level similarity score into entity matching score and context matching score . |
| Outcome: | The proposed method achieves higher matching accuracy and faster inference speed than state-of-the-art methods. |
UnifiedMLLM: Enabling Unified Representation for Multi-modal Multi-tasks With Large Language Model (2025.findings-naacl)
Copied to clipboard
Zhaowei Li, Wei Wang, YiQing Cai, Qi Xu, Pengyu Wang, Dong Zhang, Hang Song, Botian Jiang, Zhida Huang, Tao Wang
| Challenge: | Representative models like LLaVA and MiniGPT-4 have great capabilities in various tasks. |
| Approach: | They propose a unified model to represent various multi-modal tasks using a single representation. |
| Outcome: | The proposed model outperforms existing models in a variety of tasks while maintaining generality and scalability. |
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
EdgeFormer: A Parameter-Efficient Transformer for On-Device Seq2seq Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Extensive experiments show EdgeFormer can effectively outperform previous parameter-efficient Transformer baselines and achieve competitive results under both the computation and memory constraints. |
| Approach: | They propose a parameter-efficient Transformer for on-device seq2seq generation that uses two novel principles for cost-effective parameterization. |
| Outcome: | Extensive experiments show that EdgeFormer outperforms the previous parameter-efficient Transformers and achieves competitive results under both the computation and memory constraints. |
LoRAMoE: Alleviating World Knowledge Forgetting in Large Language Models via MoE-Style Plugin (2024.acl-long)
Copied to clipboard
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, Shiliang Pu, Jiang Zhu, Rui Zheng, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Experimental results show that, as the instruction data increases, LoRAMoE can significantly improve the ability to process downstream tasks, while maintaining the world knowledge stored in the LLM. |
| Approach: | They propose a framework that introduces several low-rank adapters and integrates them by using a router network to freeze the backbone model and force a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks. |
| Outcome: | The proposed framework freezes the backbone model and forces a portion of LoRAs to focus on leveraging world knowledge to solve downstream tasks, to alleviate world knowledge forgetting. |
A Lexicon-Based Graph Neural Network for Chinese NER (D19-1)
Copied to clipboard
| Challenge: | Chinese named entity recognition models are vulnerable to word ambiguities due to the lack of global semantics and chain structure. |
| Approach: | They propose a lexicon-based graph neural network with global semantics to solve word ambiguities in Chinese named entity recognition (NER) Lexicons are used to construct the graph and provide word-level features. |
| Outcome: | The proposed model improves on four NER datasets on Chinese characters, potential words, and the whole-sentence semantics. |
How Do Answer Tokens Read Reasoning Traces? Self-Reading Patterns in Thinking LLMs for Quantitative Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work on activation steering has focused on shaping reasoning traces, but it remains unclear how answer tokens actually read and integrate the reasoning to produce reliable outcomes. |
| Approach: | They propose a training-free steering method that uses self-reading quality scores to guide inference toward benign self-readiness and away from uncertain and disorganized reading. |
| Outcome: | The proposed method yields consistent accuracy gains in the reasoning traces generated by thinking LLMs. |
LoopCoder: Scaling Code Intelligence via Looped Language Models (2026.findings-acl)
Copied to clipboard
Jian Yang, Wei Zhang, Shuyue Guo, Yizhi LI, Linzheng Chai, Zhengmao Ye, Shukai Liu, Yuyang Song, Jiajun Wu, Che Liu, Tianyu Zheng, Siwei Wu, Leo L, Xudong Ma, Chuan Hao, Ran Tao, Yan Xing, Jianzhou Wang, Mingjie Tang, Aishan Liu, Zhoujun Li, Xianglong Liu, Weifeng Lv, Bryan Dai
| Challenge: | Large language models have mastered syntax-level code generation, but complex algorithmic reasoning remains a challenge. |
| Approach: | They propose a recurrent inductive bias that aligns with the recursive nature of programming logic. |
| Outcome: | The proposed model achieves comparable performance to standard dense models with more parameters. |
Data Interpreter: An LLM Agent for Data Science (2025.findings-acl)
Copied to clipboard
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Robert Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Yongxin Ni, Zhibin Gou, Zongze Xu, Yuyu Luo, Chenglin Wu
| Challenge: | Large Language Models (LLMs) excel in various domains but face challenges when applied to data science workflows due to their complex, multi-stage nature. |
| Approach: | They propose a hierarchical graph-based agent that represents complexity and a progressive strategy for step-by-step verification, refinement, and consistent context management. |
| Outcome: | The proposed agent surpasses state-of-the-art baselines on the MATH dataset and performs better on InfiAgent-DABench. |
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on compression of pretrained language models usually use preserved accuracy as the metric for evaluation. |
| Approach: | They propose two new metrics that measure how closely a compressed model mimics the original model. |
| Outcome: | The proposed metrics measure how closely a compressed model (i.e., student) mimics the original model (e.g., teacher). |
One Time of Interaction May Not Be Enough: Go Deep with an Interaction-over-Interaction Network for Response Selection in Dialogues (P19-1)
Copied to clipboard
| Challenge: | Currently, retrieval-based dialogues are performed in shallow ways . a recent study investigated the problem of context-response matching in open-domain . |
| Approach: | They propose a model that lets utterance-response interaction go deep by stacking interaction blocks. |
| Outcome: | The proposed model outperforms state-of-the-art methods on three benchmark data sets. |
MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing (2026.acl-industry)
Copied to clipboard
Junbo Niu, Zheng Liu, Zhuangcheng Gu, Bin Wang, Linke Ouyang, Zhiyuan Zhao, Tao Chu, Tianyao He, Fan Wu, Qintong Zhang, Zhenjiang Jin, Guang Liang, Rui Zhang, Wenzheng Zhang, Yuan Qu, Zhifei Ren, Yuefeng Sun, Zirui Tang, Boyu Niu, Yuanhong Zheng, Dongsheng Ma, Ziyang Miao, Hejun Dong, Siyi Qian, Junyuan Zhang, Fangdong Wang, Jingzhou Chen, Xiaomeng Zhao, Liqun Wei, Wei Li, Shasha Wang, RuiLiang Xu, Yuanyuan Cao, Lu Chen, Qianqian Wu, Huaiyu Gu, Lindong Lu, Dechen Lin, null Shenguanlin, Xuanhe Zhou, Linfeng Zhang, Yuhang Zang, Xiaoyi Dong, Jiaqi Wang, Bo Zhang, Lei Bai, Pei Chu, Weijia Li, Jiang Wu, Lijun Wu, Zhenxiang Li, Guangyu Wang, Zhongying Tu, Chao Xu, Kai Chen, Bowen Zhou, Dahua Lin, Wentao Zhang, Conghui He
| Challenge: | Document images are characterized by higher resolutions, denser content, and more complex structural layouts. |
| Approach: | They propose a 1.2B-parameter document parsing vision-language model that decouples layout analysis from local content recognition. |
| Outcome: | The proposed model surpasses general-purpose and domain-specific models on multiple benchmarks while maintaining significantly lower computational overhead. |
Blow the Dog Whistle: A Chinese Dataset for Cant Understanding with Common Sense and World Knowledge (2021.naacl-main)
Copied to clipboard
| Challenge: | Cant is important for understanding advertising, comedies and dogwhistle politics . currently, there are very few resources available for the research of cant . |
| Approach: | They propose a large and diverse dataset for creating and understanding cant from a computational linguistics perspective. |
| Outcome: | The proposed dataset can be used to test word embedding similarity and pretrained language models. |
DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing (2026.findings-acl)
Copied to clipboard
Hongzhi Zhang, Yuanze Hu, Tinghai Zhang, Jia Fu, Tao Wang, Junwei Jing, Zhaoxin Fan, Wei Bi, Ruiming Tang, Han Li, Guorui Zhou, Kun Gai
| Challenge: | Large Language Models (LLMs) are evolving towards autonomous agents . retrieval capabilities are well-benchmarked, but post-retrieval synthesis is under-evaluated due to open-ended writing. |
| Approach: | They propose a benchmark to evaluate information consolidation capabilities using survey papers as gold standards. |
| Outcome: | The proposed benchmark analyzes the post-retrieval synthesis stage of large language models . it leverages high-quality survey papers as gold standards and reverse-engineers research requests . the proposed benchmark outperforms single-turn generation and reduces hallucinations . |
Structured Pruning for Efficient Generative Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative Pre-trained Language Models (PLMs) are limited in their deployment in real-world applications. |
| Approach: | They propose to prune the feed-forward networks of generative pre-trained language models to smaller widths without designing extra operators. |
| Outcome: | The proposed method achieves 1.51x/6.96x inference speedup on GPU/CPU with 67% size reduction. |
BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solving (2025.acl-long)
Copied to clipboard
Teng Wang, Wing Yin Yu, Zhenqi He, Zehua Liu, HaileiGong HaileiGong, Han Wu, Xiongwei Han, Wei Shi, Ruifeng She, Fangzhou Zhu, Tao Zhong
| Challenge: | Existing datasets in operations research domain lack detailed annotations of the modeling process, focusing only on objective values. |
| Approach: | They propose an annotation-based tree-of-thought tree-based reasoning algorithm that integrates reinforcement learning into a tree- of-though. |
| Outcome: | The proposed algorithm outperforms state-of-the-art methods on StructuredOR, NL4OPT, and MAMO-ComplexLP datasets. |
AgentV-RL: Scaling Reward Modeling with Agentic Verifier (2026.findings-acl)
Copied to clipboard
Jiazheng Zhang, Ziche Fu, Zhiheng Xi, Wenqing Jing, Mingxu Chai, Wei He, Guoqiang Zhang, Chenghao Fan, Chenxin An, Wenxiang Chen, Zhicheng Liu, Haojie Pan, Dingwei Zhu, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing approaches to improve LLM reasoning are limited in complex domains and lack external grounding makes verifiers unreliable on computation-intensive tasks. |
| Approach: | They propose a framework that transforms reward modeling into a multi-turn, tool-augmented deliberative process. |
| Outcome: | The proposed framework surpasses state-of-the-art ORMs by 25.2% under parallel and sequential TTS. |
Unveiling and Consulting Core Experts in Retrieval-Augmented MoE-based LLMs (2024.emnlp-main)
Copied to clipboard
Xin Zhou, Ping Nie, Yiwen Guo, Haojie Wei, Zhanqiu Zhang, Pasquale Minervini, Ruotian Ma, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing research seeks to enhance RAG performance by retrieving higher-quality documents or designing RAG-specific LLMs, but internal mechanisms that contribute to RAG’s effectiveness remain underexplored. |
| Approach: | They propose to examine the internal mechanisms within the popular Mixture-of-Expert (MoE)-based LLMs and examine their ability to improve RAG by examining expert activations. |
| Outcome: | The proposed method significantly improved the ability of Large Language Models (LLMs) to solve knowledge-intensive tasks. |
Does Reasoning Introduce Bias? A Study of Social Bias Evaluation and Mitigation in LLM Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in large language models have enabled automatic generation of chain-of-thought reasoning . however, when reasoning steps reflect social stereotypes, they can reinforce harmful associations and lead to misleading conclusions. |
| Approach: | They propose a method that detects how model predictions change across incremental reasoning steps. |
| Outcome: | The proposed method outperforms a stereotype-free baseline and improves accuracy. |
“I’ve Decided to Leak”: Probing Internals Behind Prompt Leakage Intents (2025.emnlp-main)
Copied to clipboard
Jianshuo Dong, Yutong Zhang, Liu Yan, Zhenyu Zhong, Tao Wei, Ke Xu, Minlie Huang, Chao Zhang, Han Qiu
| Challenge: | Large language models (LLMs) exhibit prompt leakage vulnerabilities, raising intellectual property and confidentiality concerns. |
| Approach: | They use probing techniques to capture LLMs’ intent-related internal representations and show that they internalize prompt leakage intents in their hidden states before generating tokens. |
| Outcome: | The proposed probes achieve 90%+ AUROC across all tested models, even when applied to new system prompts and attacks. |
Compression of Generative Pre-trained Language Models via Quantization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to compress generative pre-trained language models fail on generative tasks due to homogeneous word embeddings and limited memory. |
| Approach: | They propose a token-level contrastive distillation method to learn distinguishable word embeddings and a module-wise dynamic scaling method to make quantizers adaptive to different modules. |
| Outcome: | The proposed method outperforms the state-of-the-art compression methods on generative PLMs by a clear margin. |
DeReA: Improving Idiom Translation with Detect-Retrieve-Arbitrate Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to idiom translation are limited by the constraints of static parametric memory and retrieval noise . idiomatic expressions are non-compositional units where figurative meanings diverge from literal interpretations . |
| Approach: | They propose a detect-retrieve-arbitrate framework that detects idiomatic spans by reasoning over semantic conflicts between literal and contextual meanings. |
| Outcome: | The proposed framework improves GPT-5-mini and Emerging Slang datasets on various model scales. |
Mitigating Tail Narrowing in LLM Self-Improvement via Socratic-Guided Sampling (2025.naacl-long)
Copied to clipboard
Yiwen Ding, Zhiheng Xi, Wei He, Lizhuoyuan Lizhuoyuan, Yitao Zhai, Shi Xiaowei, Xunliang Cai, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Large language models (LLMs) generate solutions themselves and iteratively train on filtered, high-quality rationales, but performance reaches a ceiling after a few iterations. |
| Approach: | They propose a strategy to improve the efficiency of sampling heavy-tailed data by using Socratic-style guidance signals to help LLMs reasoning with complex queries. |
| Outcome: | The proposed approach is effective on difficult queries and on held-out tasks, while requiring human supervision. |
AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)
Copied to clipboard
Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Xin Guo, Dingwen Yang, Chenyang Liao, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang
| Challenge: | Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents. |
| Approach: | They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
| Outcome: | The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)
Copied to clipboard
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
Learning a Matching Model with Co-teaching for Multi-turn Response Selection in Retrieval-based Dialogue Systems (P19-1)
Copied to clipboard
| Challenge: | Existing methods for learning a robust matching model from noisy training data are retrieval-based or generation-based. |
| Approach: | They propose a general co-teaching framework that learns matching models from noisy training data. |
| Outcome: | The proposed learning framework can improve existing models on two public data sets. |
Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents (2024.acl-long)
Copied to clipboard
Shihan Deng, Weikai Xu, Hongda Sun, Wei Liu, Tao Tan, Liujianfeng Liujianfeng, Ang Li, Jian Luan, Bin Wang, Rui Yan, Shuo Shang
| Challenge: | Existing benchmarks for LLM-based mobile agents are insufficient to evaluate their capabilities. |
| Approach: | They propose a benchmark to evaluate LLM-based mobile agents' planning capabilities . they expand UI operations by incorporating 103 APIs to accelerate task completion . |
| Outcome: | The proposed benchmarks are based on 103 collected APIs and real user queries . the data is categorized into three distinct groups: SAST, SAMT, and MAMT . |
Text-to-TrajVis: Enabling Trajectory Data Visualizations from Natural Language Questions (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for this task are limited and there is no suitable one available. |
| Approach: | They propose a new visualization language called Trajectory Visualization Language (TVL) to facilitate querying trajectory data and generating visualizations. |
| Outcome: | The proposed language can be used to query and generate trajectory data and generate visualizations with large language models. |
Neural Response Generation with Meta-words (P19-1)
Copied to clipboard
| Challenge: | Experimental results show that meta-words can be used to generate open domain dialogues . human-machine conversation is a fundamental problem in NLP . |
| Approach: | They propose a goal-tracking memory network that formalizes meta-word expression as a target in response generation and manages the generation process with a state memory panel and a controller. |
| Outcome: | The proposed model outperforms state-of-the-art generation models in response relevance, response diversity, and accuracy. |
MolXPT: Wrapping Molecules with Text for Generative Pre-training (2023.acl-short)
Copied to clipboard
| Challenge: | Experimental results show that Generative pre-trained Transformers (GPT) have great success in natural language processing. |
| Approach: | They propose a unified language model of text and molecules pre-trained on SMILES wrapped by text. |
| Outcome: | The proposed model outperforms strong baselines of molecular property prediction on MoleculeNet and performs comparably to the best model in text-molecule translation while using less than half of its parameters. |
Low-code LLM: Graphical User Interface over Large Language Models (2024.naacl-demo)
Copied to clipboard
Yuzhe Cai, Shaoguang Mao, Wenshan Wu, Zehua Wang, Yaobo Liang, Tao Ge, Chenfei Wu, WangYou WangYou, Ting Song, Yan Xia, Nan Duan, Furu Wei
| Challenge: | Low-code LLM is a visual programming interface that allows users to incorporate their ideas into the process without writing trivial prompts. |
| Approach: | They propose a human-LLM interaction framework that incorporates low-code visual programming interactions to achieve more controllable and stable responses. |
| Outcome: | The proposed framework enables users to incorporate ideas into the process without writing trivial prompts. |
From Imitation to Discrimination: Progressive Curriculum Learning for Robust Web Navigation (2026.findings-acl)
Copied to clipboard
| Challenge: | Text-based web agents offer computational efficiency for autonomous web navigation, yet they lack discrimination capabilities to reject plausible but incorrect elements in densely populated pages. |
| Approach: | They propose a model that uses a text-based web agent to learn to discriminate against incorrect elements in densely populated HTML and a training curriculum to synthesize diverse cross-domain tasks with strict verification. |
| Outcome: | Empirical evaluation shows that the model performs better than open-source models with 58.7% step success rate. |
LongHeads: Multi-Head Attention is Secretly a Long Context Processor (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle to process lengthy inputs due to limited length generalization and attention’s quadratic computational demands. |
| Approach: | They propose a training-free framework that allows each head to attend to important context chunks instead of allowing each head a full sentence . |
| Outcome: | The proposed framework unlocks multi-head attention's untapped potential by allowing each head to attend to important context chunks instead of the full sentence. |
Making Parameter-efficient Tuning More Efficient: A Unified Framework for Classification Tasks (2022.coling-1)
Copied to clipboard
Xin Zhou, Ruotian Ma, Yicheng Zou, Xuanting Chen, Tao Gui, Qi Zhang, Xuanjing Huang, Rui Xie, Wei Wu
| Challenge: | Large pre-trained language models (PLMs) have demonstrated superior performance in industrial applications. |
| Approach: | They propose a framework that re-uses existing parameter-efficient methods with a unified classifier. |
| Outcome: | The proposed framework improves the efficiency of existing parameter-efficient methods with a unified classifier. |
LONGAGENT: Achieving Question Answering for 128k-Token-Long Documents through Multi-Agent Collaboration (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been successful in understanding language and processing text, but their cost prohibits their practical applications. |
| Approach: | They propose a multi-agent collaboration method that breaks down lengthy documents into smaller, more manageable chunks and organizes the member agents to read their assigned chunks. |
| Outcome: | The proposed method achieves 16.42% and 1.63% accuracy gains over existing models on single-hop and multi-hop QA settings. |
ToolCoder: A Systematic Code-Empowered Tool Learning Framework for Large Language Models (2025.acl-long)
Copied to clipboard
Hanxing Ding, Shuchang Tao, Liang Pang, Zihao Wei, Jinyang Gao, Bolin Ding, Huawei Shen, Xueqi Cheng
| Challenge: | Existing approaches to tool learning rely on hand-crafted prompts and natural language reasoning, making multi-step planning difficult and lacking precise error diagnosis and reflection mechanisms. |
| Approach: | They propose a framework that reformulates tool learning as a code generation task. |
| Outcome: | The proposed framework achieves superior performance in task completion accuracy and execution reliability compared to existing approaches. |
Aspect Based Sentiment Analysis with Gated Convolutional Networks (P18-1)
Copied to clipboard
| Challenge: | Aspect-based sentiment analysis can provide more detailed information than general sentiment analysis. |
| Approach: | They propose a model based on convolutional neural networks and gating mechanisms which can selectively output the sentiment features according to the given aspect or entity. |
| Outcome: | The proposed model can selectively output sentiment features according to the given aspect or entity. |
Pretrained Language Models for Dialogue Generation with Multiple Input Sources (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Large-scale pretrained language models have achieved outstanding performance on natural language understanding tasks. |
| Approach: | They propose to fuse attention information from multiple input sources to achieve better relevance with dialogue history than simple fusion baselines. |
| Outcome: | The proposed models deliver higher relevance with dialogue history than baselines. |
Sampling Matters! An Empirical Study of Negative Sampling Strategies for Learning of Matching Models in Retrieval-based Dialogue Systems (D19-1)
Copied to clipboard
| Challenge: | Existing studies focus on constructing a matching model with sophisticated neural architectures, but do little to how to effectively learn such architectures from data. |
| Approach: | They propose to sample negative examples to automatically construct a training set for effective model learning in retrieval-based dialogue systems by using four sampling strategies. |
| Outcome: | The proposed learning method improves the performance of matching models on two benchmarks with three matching models. |
CQG: A Simple and Effective Controlled Generation Framework for Multi-hop Question Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Current models can not ensure the complexity of generated questions, so they may generate shallow questions that can be answered without multi-hop reasoning. |
| Approach: | They propose a controlled framework to generate multi-hop questions that contain key entities in multi- hop reasoning chains and a novel Transformer-based decoder to guarantee that key entities appear in the questions. |
| Outcome: | The proposed model outperforms the state-of-the-art model 25% on HotpotQA. |
Lost in Pronunciation: Detecting Chinese Offensive Language Disguised by Phonetic Cloaking Replacement (2025.emnlp-industry)
Copied to clipboard
Haotan Guo, Jianfei He, Jiayuan Ma, Hongbin Na, Zimu Wang, Haiyang Zhang, Qi Chen, Wei Wang, Zijing Shi, Tao Shen, Ling Chen
| Challenge: | Phonetic Cloaking Replacement (PCR) is a problem in content moderation in China. |
| Approach: | They organize PCR into a four-way surface-form taxonomy and compile PCR-ToxiCN, a dataset of 500 phonetically cloaked offensive posts gathered from the RedNote platform. |
| Outcome: | The proposed model achieves only an F1-score and zero-shot chain-of-thought prompting pushes performance even lower. |
Instantaneous Grammatical Error Correction with Shallow Aggressive Decoding (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve online inference efficiency of the Transformer for instantaneous Grammatical Error Correction (GEC) are sequenceto-sequence (seq2sequ) and sequenceto sequence (saq2eq) |
| Approach: | They propose a novel approach to improve the online inference efficiency of the Transformer model for instantaneous Grammatical Error Correction (GEC) it aggressively decodes as many tokens as possible in parallel instead of always decoding only one token in each step to improve computational parallelism. |
| Outcome: | The proposed approach can achieve state-of-the-art results in English and Chinese benchmarks with 10x speedup over the Transformer-big model. |
Improved Pseudo Data for Machine Translation Quality Estimation with Constrained Beam Search (2023.emnlp-main)
Copied to clipboard
Xiang Geng, Yu Zhang, Zhejian Lai, Shuaijie She, Wei Zou, Shimin Tao, Hao Yang, Jiajun Chen, Shujian Huang
| Challenge: | evaluating the quality of machine translation outputs becomes increasingly essential with the rapid development of machine language (MT). |
| Approach: | They propose to generate pseudo data using the MT model with constrained beam search (CBSQE) they propose to preserve the reference parts with high MT probabilities as correct translations . |
| Outcome: | The proposed model outperforms strong baselines in both supervised and unsupervised settings. |
Find-the-Common: A Benchmark for Explaining Visual Patterns from Images (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in Instruction-fine-tuned Vision and Language Models (IVLMs) have prompted some studies to analyze the reasoning capabilities of IVLMs. |
| Approach: | They introduce a vision and language task for Inductive Visual Reasoning that uses common attributes across visual scenes to find common answers. |
| Outcome: | The proposed model can archive with 48% accuracy on the FTC, compared with state-of-the-art models. |
Improving the Efficiency of Grammatical Error Correction with Erroneous Span Detection and Correction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to improve the efficiency of GEC are not efficient enough for GEC. |
| Approach: | They propose a language-independent approach to improve the efficiency of GEC by dividing the task into two subtasks: ESD and ESC. |
| Outcome: | The proposed approach performs comparably to conventional seq2seq approaches in English and Chinese GEC benchmarks with less than 50% time cost for inference. |
Knowledge-Grounded Dialogue Generation with Pre-trained Language Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Empirical results indicate that pre-trained language models can significantly outperform state-of-the-art methods in both automatic evaluation and human judgment. |
| Approach: | They propose to equip a pre-trained language model with a knowledge selection module to generate knowledge-grounded dialogues. |
| Outcome: | The proposed model outperforms state-of-the-art methods in evaluation and human judgment. |
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing work on LLMs that only enhance reasoning abilities, but which lack factual hallucination and slow-thinking capabilities, argues that SPP is a cognitive synergist. |
| Approach: | They propose a Solo Performance Prompting (SPP) that transforms a single LLM into a cognitive synergist by engaging in multi-turn self-collaboration with multiple personas. |
| Outcome: | The proposed model reduces factual hallucination and maintains strong reasoning abilities on three challenging tasks . |
Refining Corpora from a Model Calibration Perspective for Chinese Spelling Correction (2024.findings-acl)
Copied to clipboard
| Challenge: | Chinese Spelling Correction (CSC) lacks large-scale high-quality corpora due to labor-intensive labeling of spelling errors in real-life writing or typing scenarios. |
| Approach: | They propose to use OCR/ASR-based generation to refine Chinese Spelling Correction models on random replacement-based corpora and filter them based on prediction confidence. |
| Outcome: | The proposed model outperforms existing models on three widely-used benchmarks while significantly alleviating over-correction. |
RACE: Retrieval-augmented Commit Message Generation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to automatically generate commit messages are repetitive or redundant. |
| Approach: | They propose a retrieval-augmented neural commit message generation method which treats the retrieved similar commit as an exemplar and leverages it to generate an accurate commit message. |
| Outcome: | The proposed method outperforms baselines on a large dataset with five programming languages and can boost existing Seq2Seq models in commit message generation. |
MdEval: Massively Multilingual Code Debugging (2026.findings-acl)
Copied to clipboard
Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Jin Ke, Wei Zhang, Hualei Zhu, Shuyue Guo, Tao Sun, Jiaheng Liu, Yunlong Duan, Yu Hao, Liqun Yang, Guanglin Niu, Ge Zhang, Zhoujun Li
| Challenge: | Existing benchmarks primarily focus on Python and are limited in terms of language diversity. |
| Approach: | They propose a multilingual debugging benchmark that includes 3.9K test samples of 20 programming languages and introduces the debug instruction corpora MdEval-Instruct by injecting bugs into the correct multilingual queries and solutions. |
| Outcome: | The proposed benchmark includes 3.9K test samples of 20 programming languages and covers the automated program repair task, bug localization task, and bug identification task. |
Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering (2020.coling-main)
Copied to clipboard
| Challenge: | Existing text VQA systems generate an answer by selecting from optical character recognition (OCR) texts or a fixed vocabulary. |
| Approach: | They propose a localization-aware answer prediction network that generates the answer and predicts a bounding box as evidence of the generated answer. |
| Outcome: | The proposed network outperforms existing methods on three benchmark datasets for the text VQA task by a noticeable margin. |
PlugAT: A Plug and Play Module to Defend against Textual Adversarial Attack (2022.coling-1)
Copied to clipboard
| Challenge: | Recent studies have shown that adversarial examples can be easily fooled by adversarially perturbed examples. |
| Approach: | They propose a pluggable defense module PlugAT to provide robust predictions by adding a few trainable parameters to the model inputs while keeping the original model frozen. |
| Outcome: | The proposed model improves robustness over several strong baselines whilst training only 9.1% parameters. |
C3: A Bilingual Benchmark for Spoken Dialogue Models Exploring Challenges in Complex Conversations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent developments in spoken dialogue models have created a gap in understanding their effectiveness in comprehending and emulating human conversations. |
| Approach: | They present a benchmark dataset which comprises 1,079 instances in English and Chinese to examine their effectiveness in emulating human conversations. |
| Outcome: | The proposed model outperforms existing models in English and Chinese by using an LLM-based evaluation method that closely aligns with human judgment. |
Detecting Adversarial Samples through Sharpness of Loss Landscape (2023.findings-acl)
Copied to clipboard
Rui Zheng, Shihan Dou, Yuhao Zhou, Qin Liu, Tao Gui, Qi Zhang, Zhongyu Wei, Xuanjing Huang, Menghan Zhang
| Challenge: | Existing studies have shown that adversarial samples are more vulnerable than normal ones to textual adversarials. |
| Approach: | They propose a simple and effective sharpness-based detector that can distinguish adversarial samples by maximizing the loss increment within the region where the inference sample is located. |
| Outcome: | The proposed method outperforms previous detection methods by large margins on three text classification tasks. |
ChatMusician: Understanding and Generating Music Intrinsically with LLM (2024.findings-acl)
Copied to clipboard
Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Liumeng Xue, Ziyang Ma, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Jie Fu, Emmanouil Benetos, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
| Challenge: | Despite LLMs' impressive capabilities in musical knowledge, music reasoning remains an unsolved task. |
| Approach: | They propose an open-source large language model (LLM) that integrates intrinsic musical abilities into LLaMA2 and GPT-3.5. |
| Outcome: | The proposed model can understand and generate music with a pure text tokenizer without external multi-modal neural structures or tokenizers. |
Connectivity Patterns are Task Embeddings (2023.findings-acl)
Copied to clipboard
Zhiheng Xi, Rui Zheng, Yuansen Zhang, Xuanjing Huang, Zhongyu Wei, Minlong Peng, Mingming Sun, Qi Zhang, Tao Gui
| Challenge: | Existing methods for predicting inter-task transferability are sparse and task-specific. |
| Approach: | They propose a method that uses connectivity patterns of neurons as a unique identifier associated with a task. |
| Outcome: | The proposed method outperforms baselines in predicting inter-task transferability across data regimes and transfer settings while keeping high efficiency in computation and storage. |
Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing data for instruction-tuning are inadequate for a wide range of tasks, limiting the scope for nuanced comprehension and interactions within these domains. |
| Approach: | They propose to use Large Language Models to explore a multitude of variations or possibilities to improve instruction-tuning data by active exploration. |
| Outcome: | The proposed approach improves domain-specific instruction coverage and shows significant improvements over baselines. |
M-Ped: Multi-Prompt Ensemble Decoding for Large Language Models (2025.findings-emnlp)
Copied to clipboard
Jiaxin Guo, Daimeng Wei, Yuanchang Luo, Hengchao Shang, Zongyao Li, Jinlong Yang, Zhanglin Wu, Zhiqiang Rao, Shimin Tao, Hao Yang
| Challenge: | a new ensemble decoding approach enhances the performance of Large Language Models. |
| Approach: | They propose a multi-prompt ensemble decoding approach to enhance LLM performance . they submit n variations of prompts with X to LLMs in batch mode to decode and derive probability distributions . |
| Outcome: | The proposed method improves pass@k rates, LENS metrics and BLEU scores on diverse NLP tasks. |
Reward Modeling Requires Automatic Adjustment Based on Data Quality (2024.findings-emnlp)
Copied to clipboard
Binghai Wang, Rui Zheng, Lu Chen, Zhiheng Xi, Wei Shen, Yuhao Zhou, Dong Yan, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Reinforcement Learning from Human Feedback (RLHF) is a method for aligning language models with human values. |
| Approach: | They propose a method that automatically adjusts reward modeling based on data quality . they use preference data to train a reward model that is more aligned with human values . |
| Outcome: | The proposed method stabilizes reward model training and significantly improves alignment performance on human preference datasets. |
Open Set Relation Extraction via Unknown-Aware Training (2023.acl-long)
Copied to clipboard
Jun Zhao, Xin Zhao, WenYu Zhan, Qi Zhang, Tao Gui, Zhongyu Wei, Yun Wen Chen, Xiang Gao, Xuanjing Huang
| Challenge: | Existing supervised relation extraction methods can still misclassify unknown relations into known relations due to the lack of supervision signals. |
| Approach: | They propose a method that regularizes the model by dynamically synthesizing negative instances that can provide the missing supervision signals. |
| Outcome: | The proposed method achieves SOTA unknown relation detection without compromising the classification of known relations. |
Improving Grammatical Error Correction with Machine Translation Pairs (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to generate error-corrected sentence pairs for improving grammatical error correction are not available. |
| Approach: | They propose a method to generate error-corrected sentence pairs for improving grammatical error correction based on machine translation models of different qualities . |
| Outcome: | The proposed method can generate multiple error-corrected sentence pairs from Chinese to English text. |
You Never Know a Person, You Only Know Their Defenses: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations (2026.findings-acl)
Copied to clipboard
Hongbin Na, Zimu Wang, Zhaoming Chen, Peilin Zhou, Yining Hua, Grace Ziqi Zhou, Haiyang Zhang, Tao Shen, Wei Wang, John Torous, Shaoxiong Ji, Ling Chen
| Challenge: | Psychological defenses are strategies people use to manage distress. |
| Approach: | They propose a dialogue corpus with help seeker utterances labeled for defense level and a DMRS Co-Pilot pipeline that provides evidence-based pre-annotations. |
| Outcome: | The proposed framework reduces annotation time by 24.0% in a counterbalanced study. |
MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding (2024.findings-emnlp)
Copied to clipboard
Qinzhuo Wu, Weikai Xu, Wei Liu, Tao Tan, Liujian Liujianfeng, Ang Li, Jian Luan, Bin Wang, Shuo Shang
| Challenge: | Recent mobile AI agents based on VLMs lack basic mobile capabilities due to their pre-trained nature. |
| Approach: | They propose a mobile AI agent based on VLMs that includes additional pre-training stages to enhance both intra- and inter-UI understanding. |
| Outcome: | The proposed model outperforms existing VLMs on the Chinese mobile dataset Mobile3M . |
Learn to Memorize: Scalable Continual Learning in Semiparametric Models with Mixture-of-Neighbors Induction Memory (2025.acl-long)
Copied to clipboard
| Challenge: | Semiparametric language models (LMs) use static storage, which lacks learning capability and is disconnected from the internal information flow of the parametric models. |
| Approach: | They reconceptualize the non-parametric memory represented by kNN-LM as a learnable Mixture-of-Neighbors Induction Memory (MoNIM) this synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks . |
| Outcome: | The proposed model is a learnable Mixture-of-neighbors induction memory (MoNIM) it synergizes the induction capabilities of attention heads with the memorization strength of feed-forward networks (FFNs). |
V-GameGym: Visual Game Generation for Code Large Language Models (2026.findings-acl)
Copied to clipboard
Wei Zhang, Jian Yang, Renshuai Tao, Linzheng Chai, Shuyue Guo, Jiajun Wu, Xiaoming Chen, Ganqu Cui, Ning Ding, Xander Xu, HU Wei, Bowen Zhou
| Challenge: | Existing code-related benchmarks focus on single modality rather than visual game development. |
| Approach: | They propose a multimodal benchmark for evaluating code large language models in visual game generation that integrates a clustering-based curation methodology and a pipeline for visual code synthesis. |
| Outcome: | The proposed framework assesses code generation and visual game generation using a sandbox environment. |
Robust Lottery Tickets for Pre-trained Language Models (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-trained language models contain smaller matching subnetworks that are not robust to adversarial examples. |
| Approach: | They propose a method to find robust tickets hidden in pre-trained language models by learning binary weight masks and an adversarial loss objective to guide the search. |
| Outcome: | The proposed method improves on previous work on adversarial robustness evaluation. |
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning (2026.findings-acl)
Copied to clipboard
Haotian Luo, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Xiaochun Cao, Dacheng Tao, Naiqiang Tan, Li Shen
| Challenge: | Recent long-thought reasoning models adopt extended reasoning processes similar to how humans ponder over complex problems. |
| Approach: | They propose a model that uses RL-style fine-tuning to reduce inference overhead while maintaining accuracy. |
| Outcome: | The proposed model reduces inference overhead while maintaining accuracy. |
What Matters in Training a GPT4-Style Language Model with Multimodal Inputs? (2024.naacl-long)
Copied to clipboard
Yan Zeng, Hanbo Zhang, Jiani Zheng, Jiangnan Xia, Guoqiang Wei, Yang Wei, Yuchen Zhang, Tao Kong, Ruihua Song
| Challenge: | Recent advances in GPT-4V have demonstrated remarkable multi-modal capabilities in processing image inputs and following open-ended instructions. |
| Approach: | They propose a plug-and-play technique to enhance multi-modal LLMs . they propose 'lynx' to train multi-modal LLM models . |
| Outcome: | The proposed training strategy improves understanding accuracy and instruction-following proficiency of multi-modal models. |
Learning from LLM Agents: In-Context Generative Models for Text Casing in E-Commerce Ads (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing NER-based transformer models are expensive and lack contextual dependencies, making them less reliable when handling unseen or ad-specific terms, e.g., brand names. |
| Approach: | They propose a two-stage approach to casing correction in e-commerce ad content that leverages Chain-of-Actions to enforce content policies while accurately handling ads-specific terms. |
| Outcome: | The proposed model outperforms existing NER-based models and achieves near-LLM performance at a fraction of the cost. |
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing (2020.emnlp-main)
Copied to clipboard
| Challenge: | a novel approach to compress neural networks by progressive module replacement is proposed . a number of techniques have been proposed to compress pretraining and fine-tuning models . |
| Approach: | They propose a model compression approach that divides BERT into modules and builds their compact substitutes. |
| Outcome: | The proposed approach outperforms existing knowledge distillation approaches on GLUE benchmark . it is based on a model that divides the original BERT into several modules and builds their substitutes . |
DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translators (2024.emnlp-main)
Copied to clipboard
| Challenge: | Concatenating large language models are adapted to context-aware neural machine translation in a concatenated way . a recent paradigm shift has been witnessed in discourse-related challenges such as zero pronoun translation . |
| Approach: | They propose an alternative adaptation approach to make large language models discriminately model and utilize inter- and intra-sentence contexts. |
| Outcome: | The proposed approach outperforms concatenation mode and improves performance in discourse modeling. |
Automatic Grammatical Error Correction for Sequence-to-sequence Text Generation: An Empirical Study (P19-1)
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have a weakness: they cannot always generate sentences without grammatical errors. |
| Approach: | They propose to use automatic grammatical error correction to improve seq2seq models . they conduct experiments on machine translation, formality style transfer, sentence compression and simplification . |
| Outcome: | The proposed system can improve grammaticality of generated text and improve formal style tasks. |
Improving Sequence-to-Sequence Pre-training via Sequence Span Rewriting (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing text infilling objectives for pretrained language models require self-supervision by masking out tokens or spans in text. |
| Approach: | They propose to extend text infilling to a self-supervised sequence-to-sequence (Seq2Sequen) task. |
| Outcome: | The proposed task improves the model's performance on various natural language generation tasks. |
UER: An Open-Source Toolkit for Pre-training Models (D19-3)
Copied to clipboard
Zhe Zhao, Hui Chen, Jinbin Zhang, Xin Zhao, Tao Liu, Wei Lu, Xi Chen, Haotang Deng, Qi Ju, Xiaoyong Du
| Challenge: | Existing work on pre-training models have shown that it is important to use a framework to deploy various pre- training models efficiently. |
| Approach: | They propose an assemble-on-demand pre-training toolkit that assembles pre-trained models on demand and encapsulates them with rich modules. |
| Outcome: | The proposed framework can reproduce state-of-the-art models or develop models that remain unexplored. |
CoTrust: Privacy-Preserving Collaboration Between Large and Small Language Models in Trusted Execution Environments (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) provide powerful text generation capabilities, but accessing sensitive user inputs raises privacy concerns. |
| Approach: | They propose a privacy-preserving collaborative inference framework that combines large language models with small language models inside TEE to preserve privacy. |
| Outcome: | Experiments show that CoTrust outperforms unconstrained LLMs on multiple question answering and summarization benchmarks while maintaining strong privacy protection. |
LEAF: Large Language Diffusion Model for Time Series Forecasting (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work has applied large language models (LLMs) into time series forecasting, but they lack an understanding of holistic temporal patterns with potential error accumulation. |
| Approach: | They propose a framework that marries Larg e Langu age Diffusion Model with time series forecasting (LEAF) they propose converting time series into tokens and adopting language diffusion models to capture temporal dependencies. |
| Outcome: | The proposed framework generates future predictions with a diffusion model from a holistic view. |
Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Study (2025.coling-main)
Copied to clipboard
| Challenge: | a new benchmark evaluates video-based optical character recognition (Video OCR) performance of multi-modal models in videos . the benchmark aims to improve video LLMs' ability to extract text from video content . previous benchmarks have focused on video QA, but not video-related QA. |
| Approach: | They propose to evaluate the video OCR performance of multi-modal models in videos . they use a semi-automated approach that integrates the OCR ability of image LLMs with manual refinement . |
| Outcome: | The proposed benchmark includes 1,028 videos and 2,961 question-answer pairs . it integrates the OCR ability of image LLMs with manual refinement . |
A Model-agnostic Data Manipulation Method for Persona-based Dialogue Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for introducing explicit personas are expensive due to their expensive collection costs. |
| Approach: | They propose a data manipulation method which is model-agnostic to be packed with any persona-based dialogue generation model to improve their performance. |
| Outcome: | The proposed method is model-agnostic to be packed with any persona-based dialogue generation model to improve their performance. |
MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Experts (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize large language models for long-context inference are inefficient and consume memory. |
| Approach: | They propose a mixed-precision quantization method via mixture of experts that inputs tokens into router chunk by chunk to reduce inference overhead. |
| Outcome: | The proposed method outperforms state-of-the-art KV cache quantization methods on multiple benchmark datasets. |
Plug and Play Knowledge Distillation for kNN-LM with External Logits (2022.aacl-short)
Copied to clipboard
| Challenge: | Despite the promising evaluation results by knowledge distillation (KD) in natural language understanding (NLU) and sequence-to-sequence (seq2sequ) tasks, KD for causal language modeling (LM) remains a challenge. |
| Approach: | They propose to use external logits to improve a student's kNN-LM by leveraging teacher's knowledge at test time. |
| Outcome: | The proposed method improves a student's kNN-LM in multiple language modeling datasets and improves perplexity. |
UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction (2021.findings-acl)
Copied to clipboard
| Challenge: | Mainstream methods that ignore the diversity among keyphrases or weakly capture the relation between tasks implicitly ignore keyphrase diversity. |
| Approach: | They propose a novel end-to-end learning framework that jointly learns to extract and generate keyphrases by exploiting latent semantic relation between extraction and generation. |
| Outcome: | The proposed approach outperforms mainstream methods on a benchmarked document on keyphrase prediction. |
ALYMPICS: LLM Agents Meet Game Theory (2025.coling-main)
Copied to clipboard
| Challenge: | Alympics provides a framework for simulating human-like strategic interactions with Large Language Model (LLM) agents. |
| Approach: | They propose a framework utilizing Large Language Models (LLM) agents for empirical game theory research. |
| Outcome: | The proposed framework can be used to study human-like strategic interactions with large language model (LLM) agents in a game on the multi-round auction of scarce survival resources. |
Playing 20 Question Game with Policy-Based Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | The 20 Questions (Q20) game encourages deductive reasoning and creativity. |
| Approach: | They propose a policy-based Reinforcement Learning method which learns optimal question selection . the method is robust to noisy answers and uses a reward network to estimate the more informative reward . |
| Outcome: | The proposed method outperforms an entropy-based engineering system and has competitive performance in noisy-free simulation environment. |
DVAGen: Dynamic Vocabulary Augmented Generation (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing dynamic vocabulary approaches struggle to generalize to novel or out-of-vocabulary words, limiting their flexibility in handling diverse token combinations. |
| Approach: | They propose an open-source framework for training, evaluation, and visualization of dynamic vocabulary-augmented language models. |
| Outcome: | The proposed framework validates the effectiveness of dynamic vocabulary-augmented language models on modern LLMs and shows support for batch inference significantly improving inference throughput. |
Two Intermediate Translations Are Better Than One: Fine-tuning LLMs for Document-level Translation Refinement (2025.acl-long)
Copied to clipboard
| Challenge: | Recent research has shown that large language models (LLMs) can enhance translation quality through self-refinement. |
| Approach: | They propose to extend translation refinement from sentence-level to document-level by using document-to-document (Doc2Doc) translations. |
| Outcome: | The proposed method improves translation quality across ten translation tasks with LLaMA-3-8B-Instruct and Mistral-Nemo-Instru. |
Learning to Express in Knowledge-Grounded Conversation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing models focus on synthesizing a dialogue with proper knowledge, but neglect that the same knowledge could be expressed differently even under the same context. |
| Approach: | They propose a model that ground dialogue generation by extra knowledge by analyzing the structure of the response and the content style of each part. |
| Outcome: | The proposed model can learn the structure style defined by a few examples and generate responses in desired content style. |
Revisiting the Reliability of Language Models in Instruction-Following (2026.acl-long)
Copied to clipboard
| Challenge: | Several benchmarks have been proposed to measure instruction-following accuracy, but these scores do not translate to reliable services in real-world use. |
| Approach: | They propose a new metric reliable@k and develop an automated pipeline to generate cousin prompts. |
| Outcome: | The proposed model can be instantiated with cousin prompts and generates high-quality cousin prompt data. |
Actively Supervised Clustering for Open Relation Extraction (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for Open Relation Extraction (OpenRE) use a two-stage pipeline, which learns relation representations and assignments in the first stage, then manually labels relation for each cluster. |
| Approach: | They propose a method that performs relation learning and relation labeling simultaneously without a significant increase in human effort. |
| Outcome: | The proposed method improves existing SOTA methods by 13.8% and 10.6% on two datasets. |
Lost in the Context: Insufficient and Distracted Attention to Contexts in Preference Modeling (2025.acl-long)
Copied to clipboard
Shihan Dou, Jiayi Chen, Chenhao Huang, Feng Chen, Wei Chengzhi, Huiyuan Zheng, Shichun Liu, Yan Liu, Chenxiao Liu, Chao Xin, Lin Yan, Zongzhang Zhang, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing reward models concatenate contexts and responses, but they often ignore crucial segments of the context that are important for evaluating the response quality. |
| Approach: | They propose a reward model that evaluates the response quality based on a given context and assigns a rewards reward. |
| Outcome: | The proposed framework significantly improves preference modeling by increasing attention to relevant information within the context and achieves better generalizability. |
SCALE: Synergized Collaboration of Asymmetric Language Translation Engines (2024.findings-acl)
Copied to clipboard
| Challenge: | In this paper, we introduce SCALE, a collaborative framework that connects a compact Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine. |
| Approach: | They propose a collaborative framework that connects a Specialized Translation Model (STM) and a general-purpose Large Language Model (LLM) as one unified translation engine. |
| Outcome: | The proposed framework outperforms both LLMs and supervised models in high-resource or challenging low-resourced settings. |
K-Level Reasoning: Establishing Higher Order Beliefs in Large Language Models for Strategic Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Strategic reasoning requires Large Language Model (LLM) agents to adapt their strategies dynamically in multi-agent environments. |
| Approach: | They propose a framework that enables Large Language Models to achieve varying levels of strategic depth by recursive mechanisms that allow agents to form higher order beliefs about others' beliefs. |
| Outcome: | The proposed framework enables LLMs to achieve varying levels of strategic depth, allowing agents to form higher order beliefs—beliefs about others’ beliefs. |
Dynamic Graph Neural ODE Network for Multi-modal Emotion Recognition in Conversation (2025.coling-main)
Copied to clipboard
| Challenge: | Existing graph-based multimodal emotion recognition methods fail to capture dynamic changes in emotions. |
| Approach: | They propose a Dynamic Graph Neural Ordinary Differential Equation Network (DGODE) which combines dynamic changes of emotions to capture temporal dependencies of speakers’ emotions. |
| Outcome: | The proposed model can capture the temporal dependencies caused by dynamic changes in emotions and can improve on two publicly available multimodal emotion recognition datasets. |
Scheduled DropHead: A Regularization Method for Transformer Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | DropHead is a structured dropout method for regularizing multi-head attention . DropHed drops entire attention heads during training to prevent overfitting . |
| Approach: | They propose a structured dropout method specifically designed for regularizing multi-head attention mechanism . DropHead drops entire attention heads during training to prevent overfitting . |
| Outcome: | The proposed method can improve transformer models by 0.9 BLEU score on translation task and around 1.0 accuracy for various text classification tasks. |
Self-Demos: Eliciting Out-of-Demonstration Generalizability in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods that rely on limited demos and out-of-demonstration (OOD) queries fail when faced with out- of-demotion queries. |
| Approach: | They propose a query-aware prompting method that elicits the inherent generalizability of large language models by query-based demo generation. |
| Outcome: | The proposed method outperforms state-of-the-art methods in the OOD setting and two public math benchmarks. |
Making Harmful Behaviors Unlearnable for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are often customized by fine-tuning for the requirements of different domains. |
| Approach: | They propose a controllable training framework to make undesired behaviors unlearnable during the fine-tuning process. |
| Outcome: | The proposed framework makes undesired behaviors unlearnable during the fine-tuning process while preserving the ability to learn other information. |
Multiplex Graph Neural Network for Extractive Text Summarization (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for extractive text summarization do not consider multiple types of inter-sentential relationships, nor model intra-sententential relationships. |
| Approach: | They propose a novel method to combine different types of relationships among sentences and words to model sentence embedding. |
| Outcome: | The proposed model is compared with existing methods on CNN/DailyMail benchmark dataset to demonstrate its effectiveness. |
EventWiki: A Knowledge Base of Major Events (L18-1)
Copied to clipboard
| Challenge: | Existing knowledge bases focus on static entities such as people, locations and organizations. |
| Approach: | They propose a new knowledge base resource called EventWiki which concentrates on major events . they show that EventWiki is a very useful resource for information extraction regarding events in NLP . |
| Outcome: | The proposed resource is the first knowledge base resource of major events. |
ComRAG: Retrieval-Augmented Generation with Dynamic Vector Stores for Real-time Community Question Answering in Industry (2025.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for Community Question Answering (CQA) focus on static knowledge, limiting their applicability to real-world scenarios. |
| Approach: | They propose a retrieval-augmented generation framework for real-time industrial CQA that integrates static knowledge with dynamic historical QA pairs via a centroid-based memory mechanism. |
| Outcome: | The proposed framework outperforms baselines on three industrial CQA datasets and achieves 25.9% improvement in vector similarity, reducing latency by 8.7%–23.3%, and lowering chunk growth from 20.23% to 2.06% over iterations. |
Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition (D18-1)
Copied to clipboard
| Challenge: | a novel information network decipherment paradigm is proposed for fine-grained coordinated cross-lingual text stream alignment. |
| Approach: | They propose to use Burst Information Networks as media to represent text streams . they propose a simple yet effective information network decipherment algorithm with diverse clues . |
| Outcome: | The proposed approach outperforms existing approaches on bilingual lexicon extraction from coordinated text streams and can harvest high-quality alignments from large amounts of streaming data for endless language knowledge mining. |
WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metrics (2026.acl-long)
Copied to clipboard
| Challenge: | Web applications (web apps) are a key arena for large language models to demonstrate their code generation capabilities and commercial potential. |
| Approach: | a new benchmark for large language models (LLMs) is designed to provide real-world user requirements and generalizable evaluation metrics. |
| Outcome: | a new benchmark for large language models (LLMs) provides a real-world, generalizable, and interpretable evaluation score . the benchmark measures user requirements, expression styles and human-preference-aligned weights . a web application can be used to demonstrate its commercial potential, authors say . |
SemRegex: A Semantics-Based Approach for Generating Regular Expressions from Natural Language Specifications (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to generate programs from natural language do not address program aliasing . semantically equivalent programs may have many syntactically different forms . |
| Approach: | They propose a semantics-based approach to generate regular expressions from natural language. |
| Outcome: | The proposed approach improves on three public datasets. |
The GaoYao Benchmark: A Comprehensive Framework for Evaluating Multilingual and Multicultural Abilities of Large Language Models (2026.acl-long)
Copied to clipboard
Yilun Liu, Chunguang Zhao, Mengyao Piao, Lingqi Miao, Shimin Tao, Minggui HE, Chenxin Liu, Zhang Li, null Mahongxia, Jiaxin Guo, Chen Liu, Liqun Deng, Jiansheng Wei, Xiaojun Meng, Fanyi Du, Daimeng Wei, Yanghua Xiao
| Challenge: | Existing multilingual evaluation benchmarks neglect cultural nuances and lack language coverage in subjective tasks. |
| Approach: | They propose a framework that categorizes evaluation tasks into three cultural layers and nine cognitive sub-layers. |
| Outcome: | The proposed framework surpasses prior coverage by up to 111% on 20+ LLMs. |
Multi-Grained Knowledge Distillation for Named Entity Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained big models have delivered top performance in Seq2seq modeling, but their deployments in real-world applications are often hindered by excessive computations and memory demands. |
| Approach: | They propose a distillation scheme to efficiently transfer knowledge from big models to their cheaper counterparts. |
| Outcome: | The proposed scheme maximizes the assimilation of knowledge from the teacher model to the student model. |
OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)
Copied to clipboard
Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu
| Challenge: | a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide. |
| Approach: | They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions . |
| Outcome: | The proposed agents are based on operating systems (OS) and operating systems frameworks. |
Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show draft-then-verify paradigm can achieve around 5x speedup for the popular Transformer architectures with comparable generation quality to beam search decoding. |
| Approach: | They propose to use Spec-Drafter and Spec Verification to accelerate autoregressive (AR) decoding by combining a model optimized for efficient and accurate drafting and a reliable method for verifying the drafted tokens efficiently. |
| Outcome: | The proposed method achieves 5x speedup on seq2seq tasks with comparable generation quality to beam search decoding, refreshing the impression that draft-then-verify paradigm introduces only 1.4x2x speed up. |
UnihanLM: Coarse-to-Fine Chinese-Japanese Language Model Pretraining with the Unihan Database (2020.aacl-main)
Copied to clipboard
| Challenge: | Chinese and Japanese share many characters with similar surface morphology. |
| Approach: | They propose a Chinese-Japanese pretrained masked language model with a coarse-to-fine training approach to exploit the shared knowledge across the languages. |
| Outcome: | The proposed model is effective on mono- and cross-lingual Chinese and Japanese tasks. |
Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results prove that language models can learn from human feedback better, irrespective of sequence length . emergence of length bias often induces the model to favor longer outputs . |
| Approach: | They propose to separate reward modeling from the influence of sequence length by using the Product-of-Experts technique. |
| Outcome: | The proposed approach shows that language models perform better regardless of sequence length . the main expert is focused on understanding human intents, while the biased expert targets the identification and capture of length bias. |
CFinBench: A Comprehensive Chinese Financial Benchmark for Large Language Models (2025.naacl-long)
Copied to clipboard
Ying Nie, Binwei Yan, Tianyu Guo, Hao Liu, Haoyu Wang, Wei He, Binfan Zheng, Weihao Wang, Qiang Li, Weijian Sun, Yunhe Wang, Dacheng Tao
| Challenge: | Large language models (LLMs) have achieved remarkable performance on various NLP tasks, yet their potential in more challenging task like finance, has not been fully explored. |
| Approach: | They propose a benchmark to assess the financial knowledge of large language models (LLMs) in China. |
| Outcome: | The proposed benchmark is the most comprehensive evaluation benchmark to date for LLMs in finance. |
Distill Visual Chart Reasoning Ability from LLMs to MLLMs (2025.findings-emnlp)
Copied to clipboard
Wei He, Zhiheng Xi, Wanxu Zhao, Xiaoran Fan, Yiwen Ding, Zifei Shan, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | a new method for generating chart annotations is proposed to improve visual reasoning in multimodal large language models. |
| Approach: | They propose a code-as-intermediary translation method for distilling visual reasoning abilities from LLMs to MLLMs. |
| Outcome: | The proposed method is cost-effective, efficient and scalable. |
Route Sparse Autoencoder to Interpret Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sparse autoencoders (SAEs) extract interpretable and monosemantic features in large language models . prior work focused on feature extraction from a single layer, failing to capture activations that span multiple layers. |
| Approach: | They propose a framework that integrates a routing mechanism with a shared SAE to efficiently extract features from multiple layers. |
| Outcome: | The proposed framework extracts features from multiple layers while incurring minimal parameter overhead while achieving high interpretability and flexibility. |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Fluency Boost Learning and Inference for Neural Grammatical Error Correction (P18-1)
Copied to clipboard
| Challenge: | Seq2seq models for grammatical error correction (GEC) have two limitations: (1) a seq2q model may not be well generalized with only limited error-corrected data; (2) a model may fail to completely correct a sentence with multiple errors through normal seq1sequeq inference. |
| Approach: | They propose a fluency boost learning and inference mechanism to improve the performance of seq2seq models for grammatical error correction (GEC) by generating fluency-boost sentence pairs during training. |
| Outcome: | Experiments show that the proposed model improves on both CoNLL-2014 and JFLEG benchmark datasets. |
BERT-based Lexical Substitution (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to lexical substitution tend to overlook good substitute candidates that are not the synonyms of the target words in the lexicals and fail to take into account the substitution’s influence on the global context of the sentence. |
| Approach: | They propose an end-to-end BERT-based lexical substitution approach which proposes and validates substitute candidates without using annotated data or manually curated resources. |
| Outcome: | The proposed approach performs well in proposing and ranking substitute candidates, achieving the state-of-the-art results in both LS07 and LS14 benchmarks. |
QCRD: Quality-guided Contrastive Rationale Distillation for Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has focused on smaller, task-specific models enhanced by distilling knowledge from LLMs, but the diversity and quality of negative knowledge remains understudied. |
| Approach: | They propose a quality-guided contrastive rationale distillation framework that aims to enhance reasoning capabilities through contrastive knowledge learning. |
| Outcome: | The proposed method consistently outperforms existing distillation techniques yielding higher-quality rationales. |
Multi-grained Named Entity Recognition (P19-1)
Copied to clipboard
| Challenge: | Existing approaches treat Named Entity Recognition (NER) as a sequence labeling task. |
| Approach: | They propose a framework for Multi-Grained Named Entity Recognition where multiple entities or entity mentions in a sentence could be non-overlapping or totally nested. |
| Outcome: | The proposed framework outperforms current state-of-the-art frameworks by 4.4% in terms of the F1 score among nested/non-overlapping NER tasks. |
OntoGuard: Enforcing Action Admissibility for LLM Agents in Complex Interactive Environments (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to large language models (LLMs) are limited by their ability to enforce environmental and behavioral admissibility. |
| Approach: | They propose an ontological framework to guard LLM agents by enforcing environmental and behavioral admissibility. |
| Outcome: | Experiments on ScienceWorld and VirtualHome show that OntoGuard can enforce environmental and behavioral admissibility while preventing invalid actions. |