Papers by Tan Wang
ReCode: Robustness Evaluation of Code Generation Models (2023.acl-long)
Copied to clipboard
Shiqi Wang, Zheng Li, Haifeng Qian, Chenghao Yang, Zijian Wang, Mingyue Shang, Varun Kumar, Samson Tan, Baishakhi Ray, Parminder Bhatia, Ramesh Nallapati, Murali Krishna Ramanathan, Dan Roth, Bing Xiang
| Challenge: | Existing work on robustness in text or code tasks has focused on classification, while robustness for code generation tasks is an uncharted area. |
| Approach: | They propose a robustness evaluation benchmark for code generation models that customizes over 30 transformations specifically for code on docstrings, function and variable names, code syntax, and code format. |
| Outcome: | The proposed model performs better on human annotators and on SOTA models with human annnotators. |
We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning? (2025.acl-long)
Copied to clipboard
Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Chong Sun, Xiaoshuai Song, Jiapeng Wang, Zhuoma GongQue, Shanglin Lei, YiFan Zhang, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Xiao Zong, Yida Xu, Peiqing Yang, Zhimin Bao, Muxi Diao, Chen Li, Honggang Zhang
| Challenge: | Existing benchmarks focus more on end-to-end performance, but neglect the underlying principles of knowledge acquisition and generalization. |
| Approach: | They propose a benchmark specifically designed to explore the problem-solving principles by decomposing 6.5K visual math problems into 10.9K step-level questions for evaluation. |
| Outcome: | The proposed benchmark covers 6.5K visual math problems and 10.9K step-level questions spanning 5 layers of knowledge granularity and 67 hierarchical knowledge concepts. |
SpanProto: A Two-stage Span-based Prototypical Network for Few-shot Named Entity Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for few-shot Named Entity Recognition ignore entity boundaries and are time-consuming . a seminal span-based prototypical network solves the problem using two stages: span extraction and mention classification. |
| Approach: | They propose a seminal span-based prototypical network that tackles few-shot NER . they transform sequential tags into a global boundary matrix and use prototypical learning . |
| Outcome: | The proposed model outperforms strong baselines over multiple benchmarks. |
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method (2022.emnlp-main)
Copied to clipboard
Zeqian Ju, Peiling Lu, Xu Tan, Rui Wang, Chen Zhang, Songruoyao Wu, Kejun Zhang, Xiang-Yang Li, Tao Qin, Tie-Yan Liu
| Challenge: | a new lyric-to-melody generation system bridges the gap between lyrics and melodies . previous generation systems lack paired data and lack of control on generated melodie. |
| Approach: | They develop a lyric-to-melody generation system with music template to bridge the gap between lyrics and melodies. |
| Outcome: | The proposed system bridges the gap between lyrics and melodies by using music template. |
ReFSQL: A Retrieval-Augmentation Framework for Text-to-SQL Generation (2023.findings-emnlp)
Copied to clipboard
Kun Zhang, Xiexiong Lin, Yuanzhuo Wang, Xin Zhang, Fei Sun, Cen Jianhe, Hexiang Tan, Xuhui Jiang, Huawei Shen
| Challenge: | Existing methods that align natural language with SQL Language underestimate inherent structural characteristics of SQL and lead to structure errors. |
| Approach: | They propose a retrieval-argument framework that aligns natural language with SQL Language and trains one encoder-decoder-based model to fit all questions. |
| Outcome: | The proposed framework improves accuracy and robustness of text-to-SQL generation on five datasets. |
Fine-tuning Language Models for Joint Rewriting and Completion of Code with Potential Bugs (2024.findings-acl)
Copied to clipboard
| Challenge: | Previous work has demonstrated shortcomings of large language models of code (CodeLLMs) in completing drafty partial code with potential bugs. |
| Approach: | They propose to use large language models of code to fine-tune their models to rewrite and complete drafty partial code into functional full programs. |
| Outcome: | The proposed approach achieves superior pass rates over baselines and preserves the integrity of the original partial implementations. |
Sharing, Teaching and Aligning: Knowledgeable Transfer Learning for Cross-Lingual Machine Reading Comprehension (2023.findings-emnlp)
Copied to clipboard
| Challenge: | X-STA is a new approach for cross-lingual machine reading comprehension . the variation of answer span positions in different languages makes it difficult to transfer knowledge across languages. |
| Approach: | They propose a method that leverages an attentive teacher to subtly transfer the answer spans of the source language to the answer output space of the target. |
| Outcome: | The proposed method outperforms state-of-the-art approaches on three multi-lingual datasets. |
BlindGuard: Safeguarding LLM-based Multi-Agent Systems under Unknown Attacks (2026.acl-long)
Copied to clipboard
| Challenge: | Existing supervised defense methods rely on labeled malicious agents to train a supervised model of malicious behavior. |
| Approach: | They propose an unsupervised defense method that learns without requiring any attack-specific labels or prior knowledge of malicious behaviors. |
| Outcome: | The proposed method detects diverse attack types across MAS with various communication patterns while maintaining superior generalizability compared to baselines. |
USB: A COMPREHENSIVE AND UNIFIED SAFETY EVALUATION BENCHMARK FOR MULTIMODAL LARGE LANGUAGE MODELS (2026.acl-long)
Copied to clipboard
Baolin Zheng, Guanlin Chen, Qingyang Teng, Hongqiong Zhong, Yingshui Tan, Zhendong Liu, Weixun Wang, Jiaheng Liu, Jian Yang, Huiyun Jing, Jincheng Wei, Wenbo Su, Xiaoyong Zhu, Bo Zheng, Kaifu Zhang
| Challenge: | Existing safety benchmarks fail to provide reliable assessments due to limited risk coverage, insufficient scale and the oversight of complex modality combinations. |
| Approach: | They propose a framework that covers 61 risk categories across four modality interactions to address this gap. |
| Outcome: | The proposed framework covers 61 risk categories across four distinct modality interactions. |
Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Object hallucination has been an Achilles’ heel which hinders the broader applications of large vision-language models (LVLMs). |
| Approach: | They propose a logical closed loop-based framework for Object Hallucination Detection and Mitigation that uses logical consistency probing to raise questions with logical correlations to determine hallucinations. |
| Outcome: | The proposed method can be applied to all existing LVLMs and is effective and general. |
Extracting Multiple-Relations in One-Pass with Pre-Trained Transformers (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to extract multiple relations from a paragraph require multiple passes over the paragraph. |
| Approach: | They propose a method to extract multiple relations from a paragraph by encoding the paragraph only once. |
| Outcome: | The proposed approach can perform state-of-the-art on the benchmark ACE 2005. |
From Charts to Code: A Hierarchical Benchmark for Multimodal Models (2026.acl-long)
Copied to clipboard
Jiahao Tang, Henry Hengyuan Zhao, Lijian Wu, Zijian Zhang, Yifei Tao, Dongxing Mao, Yang Wan, Jingru Tan, Min Zeng, Min Li, Alex Jinpeng Wang
| Challenge: | Chart2Code is a new benchmark for evaluating the natural language to chart code generation capabilities of large multimodal models. |
| Approach: | They introduce Chart2Code, a new benchmark for evaluating the natural language to chart code generation capabilities of large multimodal models. |
| Outcome: | The proposed benchmark is the first to scale task complexity while capturing diverse scenarios. |
Efficient Tool Use with Chain-of-Abstraction Reasoning (2025.coling-main)
Copied to clipboard
Silin Gao, Jane Dwivedi-Yu, Ping Yu, Xiaoqing Ellen Tan, Ramakanth Pasunuru, Olga Golovneva, Koustuv Sinha, Asli Celikyilmaz, Antoine Bosselut, Tianlu Wang
| Challenge: | Recent large language models have made progress at interpreting and executing instructions. |
| Approach: | They propose a method to decouple general reasoning from specialized knowledge . they propose to use abstract reasoning chains and domain tools to reify each chain . |
| Outcome: | The proposed method outperforms baseline methods on QA and mathematical reasoning domains. |
LLM-Powered Benchmark Factory: Reliable, Generic, and Efficient (2026.acl-long)
Copied to clipboard
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Jiayi Shi, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Using generic and efficient benchmark generators, human annotators are limited by inefficiency . current benchmark generator methods rely on seed signals, leading to long cycles and high costs . |
| Approach: | They propose a framework to evaluate LLMs as generic benchmark generators and integrate them as BenchMaker. |
| Outcome: | The proposed framework achieves comparable performance to human-annotated benchmarks on most metrics. |
End-to-End Modeling via Information Tree for One-Shot Natural Language Spatial Video Grounding (2022.acl-long)
Copied to clipboard
Mengze Li, Tianbao Wang, Haoyu Zhang, Shengyu Zhang, Zhou Zhao, Jiaxu Miao, Wenqiao Zhang, Wenming Tan, Jin Wang, Peng Wang, Shiliang Pu, Fei Wu
| Challenge: | Existing methods for grounding video frames with dense annotations require enormous amount of human effort. |
| Approach: | They propose to ground natural language in video frames with only one frame labeled . they propose an end-to-end model that eliminates interference of irrelevant frames . |
| Outcome: | The proposed model can ground natural language in all video frames with only one frame labeled . the proposed model eliminates interference of irrelevant frames based on branch search and cropping techniques . |
ActionStudio: A Lightweight Framework for Data and Training of Large Action Models (2025.emnlp-main)
Copied to clipboard
Jianguo Zhang, Thai Quoc Hoang, Ming Zhu, Zuxin Liu, Shiyu Wang, Tulika Manoj Awalgaonkar, Akshara Prabhakar, Haolin Chen, Weiran Yao, Zhiwei Liu, Juntao Tan, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | Existing infrastructure for efficient agentic data processing and model training remains underdeveloped. |
| Approach: | They propose a lightweight and extensible data and training framework for large action models . they propose to unify diverse agent trajectories using Unified Format 2.0 . |
| Outcome: | The proposed framework shows 9 higher throughput than existing frameworks and performs well across public and realistic agent benchmarks. |
Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning (2025.findings-naacl)
Copied to clipboard
Xinglin Wang, Shaoxiong Feng, Yiwei Li, Peiwen Yuan, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Existing decoding strategies for chain-of-thought reasoning do not exploit prior information about question difficulty. |
| Approach: | They propose a decoding strategy called self-consistency to improve reasoning performance by adjusting the number of samples based on the posterior distribution of a set of pre-samples. |
| Outcome: | The proposed method outperforms baseline methods on arithmetic, commonsense and symbolic reasoning tasks while achieving comparable performance. |
WebClipper: Efficient Evolution of Web Agents with Graph-based Trajectory Pruning (2026.acl-long)
Copied to clipboard
Junjie Wang, Zequn Xie, Dan Yang, Jie Feng, Yue Shen, Duolin Sun, Meixiu Long, Yihan Jiao, Zhehao Tan, Jian Wang, Peng Wei, Jinjie Gu
| Challenge: | Open-source web agents rely on long tool-call trajectories with cyclic reasoning loops and exploration of unproductive branches. |
| Approach: | They propose a framework that compresses web agent trajectories via graph-based pruning. |
| Outcome: | The proposed framework reduces tool-call rounds by 20% while improving accuracy and efficiency while maintaining the same level of performance as existing models. |
Aligning Large Language Models with Implicit Preferences from User-Generated Content (2025.acl-long)
Copied to clipboard
Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang
| Challenge: | Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale. |
| Approach: | They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data. |
| Outcome: | The proposed framework transforms user-generated content into user queries and generates responses from the policy model. |
MSP: Multi-Stage Prompting for Making Pre-trained Language Models Better Translators (2022.acl-long)
Copied to clipboard
| Challenge: | Prompting has been shown to be a promising approach for applying pre-trained language models to perform downstream tasks. |
| Approach: | They propose a method that divides the translation process into three stages using pre-trained language models. |
| Outcome: | The proposed method significantly improves translation performance of pre-trained language models on three translation tasks. |
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge distillation methods rely on intermediate layer features and golden labels, which require aligned model architecture and labeled data respectively. |
| Approach: | They propose a general language model distillation method that performs two-stage word prediction distillation and vocabulary compression, which is simple and shows extremely strong performance. |
| Outcome: | The proposed method outperforms 25 state-of-the-art methods on the SuperGLUE benchmark, achieving an average score that surpasses the best method by 3%. |
Knowledgeable In-Context Tuning: Exploring and Exploiting Factual Knowledge for In-Context Learning (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have explored multiple aspects that affect the performance of large language models (LLMs) such as input-output mapping, extensive data resources, and the ability to train on labeled examples. |
| Approach: | They propose a framework that injects knowledge into LLMs during continual self-supervised pre-training and judiciously selects examples with high knowledge relevance. |
| Outcome: | The proposed framework outperforms baseline models and improves by more than 13% and 7% on text classification and question-answering tasks. |
CHIME: LLM-Assisted Hierarchical Organization of Scientific Studies for Literature Review Support (2024.findings-acl)
Copied to clipboard
Chao-Chun Hsu, Erin Bransom, Jenna Sparks, Bailey Kuehl, Chenhao Tan, David Wadden, Lucy Wang, Aakanksha Naik
| Challenge: | Literature review requires researchers to synthesize a large amount of information. |
| Approach: | They propose to use LLMs to generate hierarchical organizations from a set of studies . they use a human-in-the-loop process to correct errors in LLM-generated hierarchies . |
| Outcome: | The proposed model improves assignment of studies to categories by 12.6 F1 points. |
Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models (2025.naacl-long)
Copied to clipboard
Hengyi Wang, Haizhou Shi, Shiwei Tan, Weiyi Qin, Wenyuan Wang, Tunyu Zhang, Akshay Nambi, Tanuja Ganu, Hao Wang
| Challenge: | Multimodal Large Language Models have shown significant promise in various applications, but a comprehensive evaluation of their long-context capabilities remains underexplored. |
| Approach: | They propose a benchmark to assess the long-context capabilities of multimodal large language models. |
| Outcome: | The proposed benchmark compared MLLMs with API-based and open-source models in a long-context scenario. |
PersonaBench: Evaluating AI Models on Understanding Personal Information through Accessing (Synthetic) Private User Data (2025.findings-acl)
Copied to clipboard
Juntao Tan, Liangwei Yang, Zuxin Liu, Zhiwei Liu, Rithesh R N, Tulika Manoj Awalgaonkar, Jianguo Zhang, Weiran Yao, Ming Zhu, Shirley Kokane, Silvio Savarese, Huan Wang, Caiming Xiong, Shelby Heinecke
| Challenge: | Existing research lacks direct access to such data, making benchmarking difficult due to privacy concerns. |
| Approach: | They propose a synthetic data pipeline that generates realistic user profiles and private documents and a benchmark to evaluate models' ability to understand personal information. |
| Outcome: | The proposed pipeline generates realistic user profiles and private documents, enabling PersonaBench, a benchmark for evaluating models’ ability to understand personal information. |
Out-of-Domain Detection for Low-Resource Text Classification Tasks (D19-1)
Copied to clipboard
| Challenge: | Existing methods for OOD detection and ID classification tasks require massive amounts of ID labeled data and no OOD labeles. |
| Approach: | They propose to use OOD-resistant Prototypical Network to detect OOD cases with limited in-domain (ID) training data to solve this task. |
| Outcome: | The proposed solution outperforms state-of-the-art methods in zero-shot OOD detection task while maintaining a competitive performance on ID classification task. |
Towards Unified Prompt Tuning for Few-shot Text Classification (2022.findings-emnlp)
Copied to clipboard
Jianing Wang, Chengyu Wang, Fuli Luo, Chuanqi Tan, Minghui Qiu, Fei Yang, Qiuhui Shi, Songfang Huang, Ming Gao
| Challenge: | Prompt-based fine-tuning has boosted performance of Pre-trained Language Models (PLMs) on few-shot text classification, but PLMs are unfamiliar with prompt-style expressions during pre-training, which limits the few- shot learning performance on downstream tasks. |
| Approach: | They propose a framework for prompt-based fine-tuning that captures prompting semantics from non-target NLP datasets and propose 'Prompt-Options-Verbalizer' for joint prompt learning across different NLP tasks. |
| Outcome: | Experiments show that the proposed framework outperforms state-of-the-art prompt-based fine-tuning frameworks on few-shot text classification tasks. |
Solid-SQL: Enhanced Schema-linking based In-context Learning for Robust Text-to-SQL (2025.coling-main)
Copied to clipboard
| Challenge: | Existing text-to-SQL approaches have overlooked the critical aspect of system robustness. |
| Approach: | They propose a robust text-to-SQL solution that integrates with LLMs . their method achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% . |
| Outcome: | The proposed solution achieves SOTA SQL execution accuracy levels of 82.1% and 58.9% on the general Spider and Bird benchmarks. |
Dynamic Energy-Based Contrastive Learning with Multi-Stage Knowledge Verification for Event Causality Identification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for event causal identification rely on rule-based or random sampling strategies, which introduce spurious causal positives. |
| Approach: | They propose an ECI method enhanced by Dynamic Energy-based Contrastive Learning with multi-stage knowledge verification which generates high-quality contrastive samples and effectively suppresses spurious causal disturbances. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two benchmarks. |
Texar: A Modularized, Versatile, and Extensible Toolkit for Text Generation (P19-3)
Copied to clipboard
Zhiting Hu, Haoran Shi, Bowen Tan, Wentao Wang, Zichao Yang, Tiancheng Zhao, Junxian He, Lianhui Qin, Di Wang, Xuezhe Ma, Zhengzhong Liu, Xiaodan Liang, Wanrong Zhu, Devendra Sachan, Eric Xing
| Challenge: | Texar is an open-source text generation toolkit that supports a broad set of text generation tasks. |
| Approach: | They introduce Texar, an open-source text generation toolkit that supports text generation tasks. |
| Outcome: | Texar supports machine translation, summarization, dialog, content manipulation, and more. |
Hyperspherical Multi-Prototype with Optimal Transport for Event Argument Extraction (2024.acl-long)
Copied to clipboard
| Challenge: | Event Argument Extraction (EAE) aims to extract arguments for specified events from a text . previous work focused on long-distance dependencies of arguments, modeling co-occurrence relationships . |
| Approach: | They propose a model that takes inductive biases as targets to locate prototypes . they set multiple prototypes to represent each role to capture intra-class differences . |
| Outcome: | The proposed model achieves state-of-the-art on the RAMS and WikiEvents datasets. |
Towards Fine-Grained and Multi-Granular Contrastive Language-Speech Pre-training (2026.acl-long)
Copied to clipboard
Yifan Yang, Bing Han, Hui Wang, Wei Wang, Ziyang Ma, Long Zhou, Zengrui Jin, Guanrou Yang, Tianrui Wang, Xu Tan, Xie Chen
| Challenge: | Existing models for fine-grained speaking styles are limited in terms of accuracy, coverage, and naturalness. |
| Approach: | They propose a model that pre-trains with coarse captions and annotates with a pipeline that grounds captions in audio. |
| Outcome: | The proposed model outperforms existing models with fine-grained style annotations . it integrates global and fine-granular supervision, enabling unified representations based on the proposed model . |
Detecting Spoilers in Movie Reviews with External Movie Knowledge and User Networks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models focus on the textual content of the review, while spoiler detection requires putting the review into the context of facts and knowledge regarding movies. |
| Approach: | They propose a network-based spoiler detection model that takes into account external knowledge about movies and user activities on movie review platforms. |
| Outcome: | The proposed model takes into account external knowledge about movies and user activities on movie review platforms while incorporating user networks. |
Rethinking Depression Prediction from a Fine-Grained Subscore Modeling Perspective via Multi-Task Learning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for depression assessment rely on standardized ratings, but they are time-consuming and subject to inter-rater variability. |
| Approach: | They propose a fine-grained model for subscore prediction via multi-task learning that can be used to predict depression severity using multiple tasks. |
| Outcome: | The proposed model outperforms baselines and Qwen3-14B direct scoring on the public E-DAIC dataset and to a large-scale private clinical dataset. |
MuSe: Multi-Stage Graph Reasoning via Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Graph Neural Networks (GNNs) and graph transformers are inadequate for tasks with limited generalization. |
| Approach: | They propose a multi-stage graph reasoning framework based on vision-language models that incrementally samples and visualizes task-relevant subgraphs. |
| Outcome: | The proposed framework outperforms existing benchmarks in Graph-related tasks. |
Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies attempt to obtain optimal or suboptimal arrangements based on statistical results or using dataset-based search, but these methods increase inference overhead while leaving the model’s inherent order bias unresolved. |
| Approach: | They propose Dual Group Advantage Optimization (DGAO) which aims to improve model accuracy and order stability simultaneously. |
| Outcome: | The proposed method improves model accuracy and order stability while penalizing order-sensitive or incorrect responses. |
Semantic-Aware Action Space Compression via LLM-DRL Synergy for Efficient Task-oriented Dialogue Policy Exploration (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained large language models (LLMs) with world knowledge and semantic understanding are promising for task-oriented dialogue systems. |
| Approach: | a framework that synergizes pre-trained large language models with DRL is proposed . a lightweight action pruning mechanism is employed to eliminate implausible actions . |
| Outcome: | a new framework synergizes pre-trained large language models with DRL to guide decision-making . the proposed framework eliminates semantically implausible or low-potential actions from multi-turn dialogue context . |
LiGen: Active Lipid Generation via a Molecular Language Model (2026.acl-long)
Copied to clipboard
| Challenge: | Lipid nanoparticles (LNPs) can deliver cargos to tumor and immune cells . traditional approaches rely on experimental screening and expert judgment . |
| Approach: | They propose a method to generate lipid molecules efficiently and actively using deep learning. |
| Outcome: | The proposed method outperforms baseline methods on multiple cell lines and achieves a 30% improvement over the current methods. |
Focused Large Language Models are Stable Many-Shot Learners (2024.emnlp-main)
Copied to clipboard
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li
| Challenge: | In-Context Learning (ICL) enables large language models to achieve rapid task adaptation by learning from demonstrations. |
| Approach: | They propose a training-free method that disperses model attention from the query . they propose 'focus' search strategy that uses model perplexity to ensure sufficient attention . |
| Outcome: | The proposed method achieves an average performance improvement of 5.2% over vanilla ICL and scales well with many-shot demonstrations. |
IHEval: Evaluating Language Models on Following the Instruction Hierarchy (2025.naacl-long)
Copied to clipboard
Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, Meng Jiang
| Challenge: | Instruction-tuned language models (LMs) are increasingly deployed as interactive services across various applications. |
| Approach: | They propose a benchmark to evaluate models' ability to follow the instruction hierarchy by comparing their models to a set of benchmarks. |
| Outcome: | The proposed benchmark covers 3,538 examples across nine tasks covering cases where instructions in different priorities either align or conflict. |
InsBank: Evolving Instruction Subset for Ongoing Alignment (2025.findings-emnlp)
Copied to clipboard
Jiayi Shi, Yiwei Li, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Huan Ren, Yao Hu, Kan Li
| Challenge: | Recent studies emphasize that quality and diversity of instruction data are more crucial than quantity, highlighting the need to select diverse, high-quality subsets to reduce training costs. |
| Approach: | They propose to use a continuously updated repository to integrate the latest valuable instruction data with a progressive evolution framework to evolve InsBank over time. |
| Outcome: | The proposed framework outperforms baselines in InsBank evolution and extracts budget-specific subsets. |
Metacognitive Self-Correction for Multi-Agent System via Prototype-Guided Next-Execution Reconstruction (2026.findings-acl)
Copied to clipboard
Xu Shen, Qi Zhang, Song Wang, Zhen Tan, Xinyu Zhao, Laura Yao, Vaishnav Tadiparthi, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Kwonjoon Lee, Tianlong Chen
| Challenge: | Large Language Model based multi-agent systems (MAS) excel at collaborative problem solving but remain brittle to cascading errors. |
| Approach: | They propose a metacognitive framework that enables step-level error detection and self-correction in Large Language Model based multi-agent systems (MAS) . |
| Outcome: | The proposed framework outperforms baselines on the Who When benchmark and delivers consistent gains on AgentErrorBench. |
Generating Long-form Story Using Dynamic Hierarchical Outlining with Memory-Enhancement (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for long-form story generation rely on rigid outlines or lack macro-level planning, making it difficult to achieve contextual consistency and coherent plot development. |
| Approach: | They propose a Dynamic Hierarchical Outlining with Memory-Enhancement long-form story generation method to generate long-formed story with coherent content and plot. |
| Outcome: | The proposed method significantly improves the fluency, coherence, and overall quality of generated long stories compared to state-of-the-art methods. |
D-Artemis: A Deliberative Cognitive Framework for Mobile GUI Multi-Agents (2026.findings-acl)
Copied to clipboard
Hongze Mi, Yibo Feng, WenJie Lu, Yuqi Wang, Jinyuan Li, Song Cao, He Cui, Tengfei Tian, Xuelin Zhang, Haotian Luo, Di Sun, Jun Fang, Hua Chai, Naiqiang Tan, Gang Pan
| Challenge: | Graphical User Interface (GUI) agents aim to automate a wide spectrum of human tasks by emulating user interaction. |
| Approach: | They propose a deliberative framework that leverages a fine-grained tip retrieval mechanism to inform its decision-making process. |
| Outcome: | The proposed framework achieves SOTA among open-source general models on AndroidWorld and ScreenSpot-V2 . it leverages a fine-grained, app-specific tip retrieval mechanism to inform its decision-making process . |
OpinionDigest: A Simple Framework for Opinion Summarization (2020.acl-main)
Copied to clipboard
| Challenge: | Abstractive opinion summarization framework outperforms competitors' summarizing frameworks . extractive approaches produce well-formed text, but selecting the most popular opinions is challenging . |
| Approach: | They propose an abstractive opinion summarization framework that trains a Transformer model to reconstruct reviews from extracted opinions. |
| Outcome: | The proposed framework outperforms baselines on Yelp and shows promising customization capabilities. |
Enhancing Zero-shot and Few-shot Stance Detection with Commonsense Knowledge Graph (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for stance detection are not applicable to zero-shot and few-shot scenarios. |
| Approach: | They propose a model that integrates commonsense knowledge into a stance detection model. |
| Outcome: | The proposed model outperforms state-of-the-art methods on zero-shot and few-shot stance detection tasks. |
Knowledge Rumination for Pre-trained Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies have shown that pre-trained language models lack the capacity to handle knowledge-intensive tasks alone. |
| Approach: | They propose a new paradigm to help pre-trained language models utilize latent knowledge without retrieving it from external corpus. |
| Outcome: | The proposed paradigm can be applied to pre-trained language models without retrieving external knowledge from the corpus. |
LongRAG: A Dual-Perspective Retrieval-Augmented Generation Paradigm for Long-Context Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing long-context Large Language Models (LLMs) struggle with the “lost in the middle” issue. |
| Approach: | They propose a general, dual-perspective, and robust LLM-based RAG system paradigm for LCQA to enhance RAG’s understanding of complex long-context knowledge. |
| Outcome: | The proposed system outperforms long-context LLMs, advanced RAG, and vanilla RAG on three multi-hop datasets. |
HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-depth research on the specific capabilities needed by the RAG generation model is lacking, leading to inconsistent document quality and retrieval system imperfections. |
| Approach: | They propose that RAG models should possess three progressively hierarchical abilities: (1) Filtering: the ability to select relevant information; (2) Combination: the capability to combine semantic information across paragraphs; (3) RAG-specific reasoning: the capacity to further process external knowledge using internal knowledge. |
| Outcome: | Experiments show that the proposed method significantly improves the model’s open-book examination capability on datasets such as RGB, PopQA, MuSiQue, HotpotQA, and PubmedQA. |
In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents (2025.acl-long)
Copied to clipboard
Zhen Tan, Jun Yan, I-Hung Hsu, Rujun Han, Zifeng Wang, Long Le, Yiwen Song, Yanfei Chen, Hamid Palangi, George Lee, Anand Rajan Iyer, Tianlong Chen, Huan Liu, Chen-Yu Lee, Tomas Pfister
| Challenge: | Existing approaches to long-term dialogue memory management fail to capture the natural semantic structure of conversations, leading to fragmented and incomplete representations. |
| Approach: | They propose a mechanism that integrates forward- and backward-looking reflections into a personalized memory bank for effective future retrieval. |
| Outcome: | The proposed mechanism outperforms state-of-the-art benchmarks on a long-term dialogue memory model. |
Convex Aggregation for Opinion Summarization (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in text autoencoders have significantly improved the quality of the latent space, allowing models to generate consistent text from aggregated latent vectors. |
| Approach: | They develop a framework which searches input-output word overlap for latent vector aggregation. |
| Outcome: | The proposed framework improves the quality of the latent space and establishes state-of-the-art performance on two opinion summarization benchmarks. |
AgentAsk: Multi-Agent Systems Need to Ask (2026.acl-long)
Copied to clipboard
Bohan Lin, Kuo Yang, Zelin Tan, Yingchuan Lai, Chen Zhang, Guibin Zhang, Xinlei Yu, Miao Yu, Xu Wang, Yudong Zhang, Yang Wang
| Challenge: | Multi-agent systems fail to consistently outperform strong single-a agent baselines due to error propagation at inter-aggent message handoffs. |
| Approach: | They propose an edge-level error taxonomy that identifies four main errors in multi-agent interactions as data gaps, signal corruption, referential drift and capacity gaps as primary sources of failure. |
| Outcome: | The proposed module outperforms existing systems on five benchmarks and is architecture-agnostic. |
CLEAR: A Clinically Grounded Tabular Framework for Radiology Report Evaluation (2025.findings-emnlp)
Copied to clipboard
Yuyang Jiang, Chacha Chen, Shengyuan Wang, Feng Li, Zecong Tang, Benjamin M. Mervak, Lydia Chelala, Christopher M Straus, Reve Chahine, Samuel G. Armato Iii, Chenhao Tan
| Challenge: | Existing metrics lack the granularity and interpretability to capture nuanced clinical differences between candidate and ground-truth radiology reports. |
| Approach: | They propose a tabular framework with E**xpert-curated labels and an attribute-level comparison for radiology report evaluation (**CLEAR) |
| Outcome: | The proposed framework can extract clinical attributes and provide automated metrics that are strongly aligned with clinical judgment. |
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)
Copied to clipboard
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
Reward Difference Optimization For Sample Reweighting In Offline RLHF (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are becoming more capable, but their maximum likelihood objective for the next token prediction falls short in capturing such crucial human values. |
| Approach: | They propose a reward difference prediction method that uses reward difference coefficients to reweigh sample pairs in offline RLHF and a difference model that considers rich interactions between a pair of responses. |
| Outcome: | The proposed method is effective in both automatic metrics and human evaluation. |
Learning Easily Updated General Purpose Text Representations with Adaptable Task-Specific Prefix (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a large pre-trained language model can cause computational burdens in inference time due to multiple forward passes. |
| Approach: | They propose a method to learn fixed text representations with source tasks . they learn a task-specific prefix for each source task independently and combine them . |
| Outcome: | The proposed method improves generalizability of representations with source tasks. |
Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents (2024.acl-long)
Copied to clipboard
Shihan Deng, Weikai Xu, Hongda Sun, Wei Liu, Tao Tan, Liujianfeng Liujianfeng, Ang Li, Jian Luan, Bin Wang, Rui Yan, Shuo Shang
| Challenge: | Existing benchmarks for LLM-based mobile agents are insufficient to evaluate their capabilities. |
| Approach: | They propose a benchmark to evaluate LLM-based mobile agents' planning capabilities . they expand UI operations by incorporating 103 APIs to accelerate task completion . |
| Outcome: | The proposed benchmarks are based on 103 collected APIs and real user queries . the data is categorized into three distinct groups: SAST, SAMT, and MAMT . |
RADO: Reasoning Audit-Driven Optimization for Rigorous Reasoning in High-Stakes Domains (2026.acl-long)
Copied to clipboard
| Challenge: | Current reinforcement learning paradigms rely on outcome-based rewards, overlooking latent logical fallacies in intermediate steps. |
| Approach: | They propose a specialized audit model augmented with external tools to identify local logical ruptures and calibrate reward signals. |
| Outcome: | The proposed framework improves accuracy and logical rigor in high-stakes domains. |
RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging (2025.emnlp-main)
Copied to clipboard
Bowen Wang, Haiyuan Wan, Liwen Shi, Chen Yang, Peng He, Yue Ma, Haochen Han, Wenhao Li, Tiao Tan, Yongjian Li, Fangming Liu, Gong Yifan, Sheng Zhang
| Challenge: | Existing models that require task labels or performance trade-offs are susceptible to catastrophic forgetting. |
| Approach: | They propose a representation-aware model merging framework for continual learning without access to historical data. |
| Outcome: | The proposed framework outperforms baselines in knowledge retention and generalization across five NLP tasks and multiple continual learning scenarios. |
Agent-Pro: Learning to Evolve via Policy-Level Reflection and Optimization (2024.acl-long)
Copied to clipboard
Wenqi Zhang, Ke Tang, Hai Wu, Mengna Wang, Yongliang Shen, Guiyang Hou, Zeqi Tan, Peng Li, Yueting Zhuang, Weiming Lu
| Challenge: | Large Language Models (LLMs) are designed as specific task solvers with sophisticated prompt engineering, but are inherently incapacitating to address complex dynamic scenarios. |
| Approach: | They propose an LLM-based agent with policy-level reflection and optimization that can learn from interactive experiences and progressively elevate its behavioral policy. |
| Outcome: | The proposed agent outperforms vanilla LLM and specialized models in blackjack and Texas hold’em. |
Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Models (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit impressive performance across various domains but struggle with arithmetic reasoning tasks. |
| Approach: | They propose a Teaching-Inspired Integrated Prompting Framework which emulates the instructional process of a teacher guiding students. |
| Outcome: | The proposed framework improves reasoning accuracy on nine benchmarks. |
Personalized Benchmarking: Evaluating LLMs by Individual Preferences (2026.findings-acl)
Copied to clipboard
| Challenge: | Current benchmarks average preferences across all users to compute aggregate ratings . this overlooks individual user preferences when establishing model rankings . |
| Approach: | They compute personalized model rankings using ELO ratings and Bradley-Terry coefficients . they find users exhibit substantial heterogeneity in topical interests and communication styles . |
| Outcome: | The results show that individual rankings of LLM models diverge dramatically from aggregate rankings . a compact combination of topic and style features provides a useful feature space . |
See the World, Discover Knowledge: A Chinese Factuality Evaluation for Large Vision Language Models (2025.findings-acl)
Copied to clipboard
Jihao Gu, Yingyao Wang, Pi Bu, Chen Wang, Ziming Wang, Tengtao Song, Donglai Wei, Jiale Yuan, Yingxiu Zhao, Yancheng He, Shilong Li, Jiaheng Liu, Meng Cao, Jun Song, Yingshui Tan, Xiang Li, Wenbo Su, Xiaoyong Zhu, Bo Zheng
| Challenge: | Existing models for large vision language models do not fully reflect their knowledge capacity and reliability, resulting in erroneous outputs that do not align with the image content or provide answers lacking knowledge evidence. |
| Approach: | They propose a Chinese-based benchmark for visual factuality across 8 major topics and 56 subtopics and a multi-hop question construction. |
| Outcome: | The proposed model decouples visual factuality into two parts: seeing the world and discovering knowledge. |
Leibniz: Theory-of-Mind Driven Neuro-Symbolic Logical Reasoning via Multi-Agent Collaboration (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for logical reasoning with large language models suffer from insufficient rule semantic grounding and weak rule application mechanisms. |
| Approach: | They propose a theory-of-mind driven neuro-symbolic reasoning framework that integrates natural language and symbolic representations throughout the reasoning process. |
| Outcome: | The proposed model surpasses state-of-the-art models in reasoning accuracy and flexibility. |
Repulsive Attention: Rethinking Multi-head Attention as Bayesian Inference (2020.emnlp-main)
Copied to clipboard
Bang An, Jie Lyu, Zhenyi Wang, Chunyuan Li, Changwei Hu, Fei Tan, Ruiyi Zhang, Yifan Hu, Changyou Chen
| Challenge: | Existing studies show that multi-head attention is an effective module in deep neural networks, but there are no explicit mechanisms guaranteeing this property. |
| Approach: | They propose a non-parametric approach that explicitly improves the repulsiveness in multi-head attention and consequently strengthens model’s expressiveness. |
| Outcome: | The proposed approach improves the repulsiveness in multi-head attention and strengthens model’s expressiveness. |
InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance (2024.emnlp-main)
Copied to clipboard
Pengyu Wang, Dong Zhang, Linyang Li, Chenkun Tan, Xinghao Wang, Mozhi Zhang, Ke Ren, Botian Jiang, Xipeng Qiu
| Challenge: | Existing methods for enhancing harmlessness and helpfulness of large language models (LLMs) involve complex and resource-intensive training processes. |
| Approach: | They propose a method that decouples harmlessness from helpfulness during inference phase. |
| Outcome: | The proposed method significantly reduces the attack success rate (ASR) of harmful instructions and jailbreak instructions while maintaining almost unchanged performance in downstream tasks. |
GradOT: Training-free Gradient-preserving Offsite-tuning for Large Language Models (2025.acl-long)
Copied to clipboard
Kai Yao, Zhaorui Tan, Penglei Gao, Lichun Li, Kaixin Wu, Yinggui Wang, Yuan Zhao, Yixin Ji, Jianke Zhu, Wei Wang
| Challenge: | Existing methods for offsite-tuning of large language models require high computational costs and lack theoretical analysis. |
| Approach: | They propose an offsite-tuning approach that selectively applies compression techniques such as rank compression and channel pruning to preserve the gradients of fine-tuned adapters while ensuring privacy. |
| Outcome: | The proposed method surpasses existing OT methods in privacy protection and model performance. |
Noise-Robust Fine-Tuning of Pretrained Language Models via External Guidance (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained Language Models (PLMs) are advanced but data labels are noisy due to the complex annotation process. |
| Approach: | They propose a framework for fine-tuning PLMs using noisy labels that incorporates guidance from Large Language Models like ChatGPT. |
| Outcome: | Experiments on synthetic and real-world noisy datasets show that the proposed framework outperforms the state-of-the-art framework. |
Active Learning for Abstractive Text Summarization via LLM-Determined Curriculum and Certainty Gain Maximization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Abstractive text summarization (ATS) requires laborious data annotation and time-consuming model training. |
| Approach: | They propose a novel active learning framework that asks large language models to rate difficulty of instances and then uses certainty gain maximization to select instances with a distribution that aligns well with the overall distribution. |
| Outcome: | The proposed framework improves stability, effectiveness, and efficiency of abstractive text summarization backbones. |
DeepWell-Adol: A Scalable Expert-Based Dialogue Corpus for Adolescent Positive Mental Health and Wellbeing Promotion (2025.emnlp-main)
Copied to clipboard
| Challenge: | Promoting positive mental health and well-being is a critical yet underexplored area in natural language processing. |
| Approach: | They propose a Chinese dialogue corpus grounded in positive psychology and coaching that integrates human expert-written seed data with automated data augmentation to ensure high quality and scalability. |
| Outcome: | The proposed corpus meets general standards for psychological dialogue and emotional support while also showing superior performance across multiple models in promoting positive psychological processes, character strengths, interpersonal relationships, and healthy behaviors. |
FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition (2021.findings-emnlp)
Copied to clipboard
Yichong Leng, Xu Tan, Rui Wang, Linchen Zhu, Jin Xu, Wenjie Liu, Linquan Liu, Xiang-Yang Li, Tao Qin, Edward Lin, Tie-Yan Liu
| Challenge: | Error correction is widely used in automatic speech recognition (ASR) to post-process the generated sentence. |
| Approach: | They propose a fast correction model that takes multiple ASR candidates as input for better correction accuracy. |
| Outcome: | The proposed model can reduce the word error rate (WER) with multiple candidates by 3.2% and 2.6%. |
LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models (2026.acl-long)
Copied to clipboard
Ming Zhang, Yujiong Shen, Jingyi Deng, Yuhui Wang, Huayu Sha, Kexin Tan, Qiyuan Peng, Yue Zhang, Junzhe Wang, Shichun Liu, Yueyuan Huang, Jingqi Tong, Changhao Jiang, Yilong Wu, Zhihao Zhang, Mingqi Wu, Mingxu Chai, Zhiheng Xi, Shihan Dou, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing evaluation of Large Language Models on static benchmarks is vulnerable to data contamination and leaderboard overfitting. |
| Approach: | LLMEval-Fair framework provides a framework for dynamic evaluation of Large Language Models . evaluators use a proprietary bank of 220k graduate-level questions to analyze model data . |
| Outcome: | LLMEval-Fair provides robust and credible evaluation framework for Large Language Models . it provides a strong empirical validation for the dynamic evaluation paradigm . |
Improving Sequential Model Editing with Fact Retrieval (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to fix erroneous knowledge in Pre-trained Language models experience a performance decline when the number of edits increases. |
| Approach: | They propose a framework that leverages factual information to enhance editing generalization and guide the identification of edits by retrieving related facts from the fact-patch memory. |
| Outcome: | The proposed framework can improve model generalization and accuracy even with thousands of edits. |
Explaining Language Model Predictions with High-Impact Concepts (2024.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods to explain large language models (LLMs) are mostly correlational and lack causal features due to compositional nature of languages. |
| Approach: | They propose a framework to provide impact-aware explanations for large language models that are robust to feature changes and influential to the model’s predictions. |
| Outcome: | The proposed explanations improve on real and synthetic tasks and are robust to feature changes and influential to the model’s predictions. |
Suggest-Verify-Revise: A Three-Stage Document-Level Event Causality Identification with Narrative Consistency (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for document-level Event Causality Identification rely on local semantic similarity for independent event-pair discrimination . Existing approaches ignore the influence of the overall narrative backbone in the propagation of causal dependencies and the role differentiation of events within multi-cause/multi-effect structures. |
| Approach: | They propose a suggest-verify-revise approach for document-level Event Causality Identification with narrative consistency (SVRECI) they integrate heuristic causal suggestions generated by an LLM with structural suggestions derived from hypergraph modeling . |
| Outcome: | The proposed approach outperforms existing methods on event-storylines and Causal-TimeBank datasets. |
CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models (2025.findings-naacl)
Copied to clipboard
Shangda Wu, Yashan Wang, Ruibin Yuan, Guo Zhancheng, Xu Tan, Ge Zhang, Monan Zhou, Jing Chen, Xuefeng Mu, Yuejie Gao, Yuanliang Dong, Jiafeng Liu, Xiaobing Li, Feng Yu, Maosong Sun
| Challenge: | Current music information retrieval systems struggle to meet linguistic diversity challenges . current systems struggle with text queries in non-English languages . |
| Approach: | They propose a music information retrieval system that supports both ABC notation and MIDI . CLaMP 2 includes a multilingual text encoder and a multiple-modal music encoder . |
| Outcome: | The proposed system achieves state-of-the-art results in multilingual semantic search and music classification across modalities. |
TopKG: Target-oriented Dialog via Global Planning on Knowledge Graph (2022.coling-1)
Copied to clipboard
| Challenge: | Existing target-oriented dialogs take a local and greedy strategy for response generation, where global planning is absent. |
| Approach: | They propose a global planning method for target-oriented dialog on a commonsense knowledge graph to adjust local response generation towards the global target. |
| Outcome: | The proposed method can reach the target with a higher success rate, fewer turns, and more coherent responses. |
TabPrompt: Graph-based Pre-training and Prompting for Few-shot Table Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods of Table Understanding (TU) focus on the textual content within the tabular data, disregarding the topological information of the table. |
| Approach: | They propose a framework that uses tabs to understand tabular data without ignoring the topological information of the table. |
| Outcome: | The proposed framework outperforms baselines in few-shot table understanding tasks. |
Beyond One-Size-Fits-All: Tailored Benchmarks for Efficient Evaluation (2025.acl-long)
Copied to clipboard
Peiwen Yuan, Yueqi Zhang, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Jiayi Shi, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Existing efficient methods estimate performance of models on large benchmarks, but these methods rely on the assumption that target models have high prediction consistency with source models. |
| Approach: | They propose a method that conducts customized evaluation tailored to each target model. |
| Outcome: | The proposed method reduces the MAE of estimates by 31.4% on benchmarks across 300 models. |
Open-World Authorship Attribution (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models do not evaluate their performance in academic research . authors aim to identify authors from anonymous text without additional information . |
| Approach: | They propose a benchmark to quantitatively assess LLMs' ability to infer author from text . they propose 'open-world' authorship attribute' to be a two-stage framework . |
| Outcome: | The proposed approach achieves 60.7% accuracy and 44.3% accuracy in two stages. |
What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time (2026.acl-long)
Copied to clipboard
| Challenge: | Existing TTRL methods rely on positive pseudo-labeling strategies to enhance reasoning capabilities. |
| Approach: | They propose a test-time reinforcement learning framework that mitigates label noise amplification by deriving pseudo-rewards from majority voting consensus. |
| Outcome: | The proposed framework mitigates label noise amplification by implementing selective positive pseudo-labeling and entropy-gated negative p-labeled pruning. |
MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization (2024.findings-acl)
Copied to clipboard
Zhiyu Yang, Zihan Zhou, Shuo Wang, Xin Cong, Xu Han, Yukun Yan, Zhenghao Liu, Zhixing Tan, Pengyuan Liu, Dong Yu, Zhiyuan Liu, Xiaodong Shi, Maosong Sun
| Challenge: | Scientific data visualization is an essential process in research, but its use of large language models remains unexplored. |
| Approach: | They propose a model-agnostic LLM agent framework to automate scientific data visualization tasks. |
| Outcome: | The proposed framework improves performance of commercial and open-source models. |
Consultant Decoding: Yet Another Synergistic Mechanism (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have attracted widespread attention and adoption across diverse domains due to their exceptional performance and robust generalization abilities. |
| Approach: | They propose a synergetic mechanism for Consultant Decoding (CD) that achieves a 2.5-fold increase in inference speed compared to the target model while maintaining comparable generation quality. |
| Outcome: | The proposed mechanism achieves 2.5-fold increase in inference speed while maintaining comparable generation quality (100% of the target model’s performance). |
Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens (2026.findings-acl)
Copied to clipboard
Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang, Yancheng Wang, Yingzhen Yang, Huan Liu
| Challenge: | Chain-of-Thought (CoT) prompting has been shown to be effective in eliciting structured reasoning from large language models (LLMs). |
| Approach: | They propose a data distribution lens to understand when and why CoT reasoning fails . they propose 'data-based' training that trains LLMs from scratch . |
| Outcome: | The proposed model enables models to generate reasoning trajectories that approximate those observed during training. |
CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a global audience, so alignment must extend to cultural resonance. |
| Approach: | They propose a framework that frames alignment as a conditional capacity separation problem. |
| Outcome: | The proposed framework outperforms both dense baselines and semantic-only MoEs on three large language models. |
Learning from Diverse Reasoning Paths with Routing and Collaboration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies suggest that the reasoning abilities of large language models (LLMs) grows with model size and pre-training data. |
| Approach: | They propose to combine quality filtering, conditional routing, and cooperative peer teaching to transfer knowledge from powerful teacher models to compact and transparent students. |
| Outcome: | Experiments show that QR-Distill is superior to traditional methods. |
OmniFlatten: An End-to-end GPT Model for Seamless Voice Conversation (2025.acl-long)
Copied to clipboard
Qinglin Zhang, Luyao Cheng, Chong Deng, Qian Chen, Wen Wang, Siqi Zheng, Jiaqing Liu, Hai Yu, Chao-Hong Tan, Zhihao Du, ShiLiang Zhang
| Challenge: | Full-duplex spoken dialogue systems allow simultaneous bidirectional communication . low latency and natural interactions in full-duplice systems remains a challenge . |
| Approach: | They propose a multi-stage post-training scheme that adapts a text large language model into a speech-text dialogue LLM. |
| Outcome: | The proposed model can model human conversation behaviors with low latency and natural interactions with low delay. |
Towards Linear Time Neural Machine Translation with Capsule Networks (D19-1)
Copied to clipboard
| Challenge: | Neural Machine Translation (NMT) is an endto-end learning approach to machine translation. |
| Approach: | They propose a capsule network with dynamic routing for linear time Neural Machine Translation . they map the source sentence into a matrix with pre-determined size and apply a deep LSTM network to decode the target sequence from the source representation. |
| Outcome: | The proposed network achieves comparable results with the Transformer system on English-German and English-French tasks. |
Calibrating Inference Time Alignment with Sequence-level Risk Accumulation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to decode large language models (LLMs) often over-reject benign information, limiting their generalizability in real-world scenarios where harmful and benign information coexist. |
| Approach: | They propose a framework to regulate decoding alignments for Large Language Models (LLMs) they employ a reward-guided branch decoding paradigm to incorporate safety awareness during generation. |
| Outcome: | The proposed framework achieves superior performance on four attack benchmarks and two neutral datasets. |
MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to align large language models with human preferences often result in a static alignment that cannot account for the diversity of human preferences in practical applications. |
| Approach: | They propose a method to help large language models dynamically align with various explicit or implicit preferences specified at inference time. |
| Outcome: | The proposed method can help LLMs dynamically align with various explicit or implicit preferences specified at the inference stage, validating the feasibility of MetaAlign. |
Fundamental Reasoning Paradigms Induce Out-of-Domain Generalization in Language Models (2026.findings-acl)
Copied to clipboard
Mingzi Cao, Xingwei Tan, Mahmud Elahi Akhter, Marco Valentino, Maria Liakata, Xi Wang, Nikolaos Aletras
| Challenge: | Deduction, induction, and abduction are fundamental reasoning paradigms, core for human logical thinking. |
| Approach: | They propose to use a dataset of symbolic tasks to induce deductive skills into large language models (LLMs) they then use FT to fine-tune models to improve OOD generalization . |
| Outcome: | The proposed approach yields strong generalizability with substantial performance gains (up to 14.60) across realistic out-of-domain tasks. |
Integrating Vectorized Lexical Constraints for Neural Machine Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing studies focus on integrating discrete lexical constraints into neural machine translation models. |
| Approach: | They propose to integrate constraints into NMT models by integrating them into keys and values . they show that their method outperforms representative baselines on four language pairs . |
| Outcome: | The proposed method outperforms baselines on four language pairs, showing superiority . |
MobileVLM: A Vision-Language Model for Better Intra- and Inter-UI Understanding (2024.findings-emnlp)
Copied to clipboard
Qinzhuo Wu, Weikai Xu, Wei Liu, Tao Tan, Liujian Liujianfeng, Ang Li, Jian Luan, Bin Wang, Shuo Shang
| Challenge: | Recent mobile AI agents based on VLMs lack basic mobile capabilities due to their pre-trained nature. |
| Approach: | They propose a mobile AI agent based on VLMs that includes additional pre-training stages to enhance both intra- and inter-UI understanding. |
| Outcome: | The proposed model outperforms existing VLMs on the Chinese mobile dataset Mobile3M . |
Variational Language Concepts for Interpreting Foundation Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Foundation Language Models (FLMs) have achieved remarkable success in natural language processing. |
| Approach: | They propose a variational Bayesian framework to provide word-level interpretations for FLMs . they propose valc to find optimal language concepts to interpret FLM predictions . |
| Outcome: | Empirical results show that the proposed framework can provide conceptual interpretations for foundation language models. |
Decoupled Proxy Alignment: Mitigating Language Prior Conflict for Multimodal Alignment in MLLMs (2025.findings-emnlp)
Copied to clipboard
Chenkun Tan, Pengyu Wang, Shaojun Zhou, Botian Jiang, Zhaowei Li, Dong Zhang, Xinghao Wang, Yaqian Zhou, Xipeng Qiu
| Challenge: | Recent advances in multimodal large language models focus on improving performance . however, language prior conflict leads to suboptimal vision-language alignment . |
| Approach: | They propose a method to decouple the alignment process from language prior interference . they use a proxy LLM to detach from language interference during pretraining . |
| Outcome: | The proposed method improves training performance and generalizes training data. |
Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Fengran Mo, Yifan Gao, Sha Li, Hansi Zeng, Xin Liu, Zhaoxuan Tan, Xian Li, Jianshu Chen, Dakuo Wang, Meng Jiang
| Challenge: | Existing studies focus on single-turn scenarios, which might lack the ability to handle multi-turn interactions. |
| Approach: | They propose a conversational agent that interleaves search and reasoning across turns and provides tailored rewards towards evolving user goals. |
| Outcome: | The proposed agent interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training with tailored rewards towards evolving user goals. |
Domaino1s: Guiding LLM Reasoning for Explainable Answers in High-Stakes Domains (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) generate brief answers without reasoning processes and explanations. |
| Approach: | They propose supervised fine-tuning and tree search to enhance LLMs’ reasoning capabilities on domain tasks. |
| Outcome: | The proposed model improves on stock investment recommendation and legal reasoning QA tasks. |
PsyMem: Fine-grained Psychological Alignment and Explicit Memory Control for Advanced Role-Playing LLMs (2026.tacl-1)
Copied to clipboard
| Challenge: | Existing role-playing models rely on superficial textual descriptions or simplistic metrics, inadequately modeling both intrinsic and extrinsic character dimensions. |
| Approach: | They propose a framework that integrates fine-grained psychological attributes and explicit memory control for role-playing. |
| Outcome: | The proposed framework outperforms baseline models in human-likeness and character fidelity. |
MuggleMath: Assessing the Impact of Query and Response Augmentation on Math Reasoning (2024.acl-long)
Copied to clipboard
Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, Chang Zhou
| Challenge: | In math reasoning with large language models, fine-tuning data augmentation by query evolution and diverse reasoning paths is empirically verified effective. |
| Approach: | They propose to fine-tune data augmentation by query evolution and diverse reasoning paths. |
| Outcome: | The proposed model achieves new state-of-the-art on GSM8K and MATH. |
DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models are limited by challenges in factuality and hallucinations to be directly employed off-the-shelf for judging the veracity of news articles. |
| Approach: | They propose to integrate large language models into the news pipeline by generating news reactions and generating proxy tasks. |
| Outcome: | The proposed model outperforms state-of-the-art baselines by 16.8% in macro f1-score on seven datasets with three LLMs. |
MTGP: Multi-turn Target-oriented Dialogue Guided by Generative Global Path with Flexible Turns (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches focus on global planning, which plans toward the target before the conversation. |
| Approach: | They propose to generate a global path as a natural language sentence instead of a sequence of nodes. |
| Outcome: | The proposed method has fewer turns, more coherent semantics, and higher success rate than baselines. |
Too Consistent to Detect: A Study of Self-Consistent Errors in LLMs (2025.emnlp-main)
Copied to clipboard
Hexiang Tan, Fei Sun, Sha Liu, Du Su, Qi Cao, Xin Chen, Jingang Wang, Xunliang Cai, Yuanzhuo Wang, Huawei Shen, Xueqi Cheng
| Challenge: | Existing detection methods fail to account for **self-consistent error** . study identifies self-consistency errors and evaluates them . |
| Approach: | They propose a method that fuses hidden state evidence from an external verifier LLM to detect self-consistent errors. |
| Outcome: | The proposed method significantly enhances performance on self-consistent errors across three LLM families. |
Self-Generated Critiques Boost Reward Modeling for Language Models (2025.naacl-long)
Copied to clipboard
Yue Yu, Zhengxing Chen, Aston Zhang, Liang Tan, Chenguang Zhu, Richard Yuanzhe Pang, Yundi Qian, Xuewei Wang, Suchin Gururangan, Chao Zhang, Melanie Kambadur, Dhruv Mahajan, Rui Hou
| Challenge: | Existing reward models produce scalar scores and struggle to incorporate critiques in a natural language format. |
| Approach: | They propose a framework that predicts critiques and rewards using self-generated critiques without extra supervision. |
| Outcome: | The proposed framework improves reward modeling accuracy by 3.7%-7.3% compared to standard reward models and LLM judges. |
O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning (2026.findings-acl)
Copied to clipboard
Haotian Luo, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Xiaochun Cao, Dacheng Tao, Naiqiang Tan, Li Shen
| Challenge: | Recent long-thought reasoning models adopt extended reasoning processes similar to how humans ponder over complex problems. |
| Approach: | They propose a model that uses RL-style fine-tuning to reduce inference overhead while maintaining accuracy. |
| Outcome: | The proposed model reduces inference overhead while maintaining accuracy. |
Blinded by Generated Contexts: How Language Models Merge Generated and Retrieved Contexts When Knowledge Conflicts? (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in augmenting Large Language Models (LLMs) with auxiliary information have significantly revolutionized their efficacy in knowledge-intensive tasks. |
| Approach: | They propose a systematic framework to identify whether LLMs’ responses are attributed to either generated or retrieved contexts. |
| Outcome: | The proposed framework identifies whether LLMs’ responses are attributed to either generated or retrieved contexts. |
Towards Adaptive Prefix Tuning for Parameter-Efficient Language Model Fine-tuning (2023.acl-short)
Copied to clipboard
| Challenge: | Parameter-efficient fine-tuning only optimizes a few task-specific parameters with frozen pre-trained model. |
| Approach: | They propose to optimize a prefix vector inserted into Transformer layers to optimize the prefix . they propose to use a gate mechanism to adjust the prefixed to each layer . |
| Outcome: | The proposed approach improves on the SuperGLUE and NER datasets. |
Glue pizza and eat rocks - Exploiting Vulnerabilities in Retrieval-Augmented Generative Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Retrieval-Augmented Generative (RAG) models enhance Large Language Models (LLMs) by integrating external knowledge bases. |
| Approach: | They propose to exploit openness of RAG models by injecting deceptive content into the retrieval database, intentionally changing the model’s behavior. |
| Outcome: | The proposed model can be exploited through crafted content uploads with access to the retriever. |
Learning from LLM Agents: In-Context Generative Models for Text Casing in E-Commerce Ads (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing NER-based transformer models are expensive and lack contextual dependencies, making them less reliable when handling unseen or ad-specific terms, e.g., brand names. |
| Approach: | They propose a two-stage approach to casing correction in e-commerce ad content that leverages Chain-of-Actions to enforce content policies while accurately handling ads-specific terms. |
| Outcome: | The proposed model outperforms existing NER-based models and achieves near-LLM performance at a fraction of the cost. |
What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection (2024.acl-long)
Copied to clipboard
| Challenge: | Social media bot detection has always been an arms race between advancements in machine learning and adversarial bot strategies to evade detection. |
| Approach: | They propose a mixture-of-heterogeneous-experts framework to divide and conquer diverse user information modalities and propose LLM-guided manipulation of user textual and structured information to evade detection. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines on 1,000 annotated examples while bringing down existing detectors by 29.6% and harming calibration and reliability of bot detection systems. |
NUMCoT: Numerals and Units of Measurement in Chain-of-Thought Reasoning using Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing LLMs are not able to handle numerals and units of measurement, but they can be improved by introducing perturbations. |
| Approach: | They propose to analyze existing LLMs on processing numerals and units of measurement by perturbing their datasets. |
| Outcome: | The proposed model improves on ancient Chinese arithmetic problems and can handle numeral and measurement conversions. |
SATBench: Benchmarking LLMs’ Logical Reasoning via Automated Puzzle Generation from SAT Formulas (2025.emnlp-main)
Copied to clipboard
Anjiang Wei, Yuheng Wu, Yingjia Wan, Tarun Suresh, Huanmi Tan, Zhanke Zhou, Sanmi Koyejo, Ke Wang, Alex Aiken
| Challenge: | SATBench is a benchmark for evaluating the logical reasoning capabilities of large language models (LLMs) through logical puzzles derived from Boolean satisfiability (SAT) problems. |
| Approach: | They propose a benchmark to evaluate logical reasoning capabilities of large language models (LLMs) using logical puzzles derived from Boolean satisfiability problems. |
| Outcome: | The proposed model achieves 65.0% accuracy on hard UNSAT problems, close to the baseline of 50%. |
Context-Aware Conversation Thread Detection in Multi-Party Chat (D19-1)
Copied to clipboard
| Challenge: | In multi-party chat, it is common for multiple conversations to occur concurrently . a new model that automatically disentangles conversation threads is proposed . |
| Approach: | They propose a Context-Aware Thread Detection model that automatically disentangles conversation threads in chat logs. |
| Outcome: | The proposed model outperforms state-of-the-art models on four real-world chat logs. |
OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain (2025.emnlp-main)
Copied to clipboard
| Challenge: | a new benchmark for RAG is developed for the financial domain . omnidirectional and automatic benchmarks are difficult to build in vertical domains . |
| Approach: | They propose an omnidirectional and automatic RAG benchmark for the financial domain . they categorize RAG scenarios by task classes and 16 financial topics . |
| Outcome: | The proposed benchmark achieves an 87.47% acceptance ratio in human evaluations of generated instances. |
LATTE: Learning to Think with Vision Specialists (2025.emnlp-main)
Copied to clipboard
Zixian Ma, Jianguo Zhang, Zhiwei Liu, Jieyu Zhang, Juntao Tan, Manli Shu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Caiming Xiong, Ranjay Krishna, Silvio Savarese
| Challenge: | Open-source vision-language models excel on simple question-answering tasks, but struggle with complex questions that require both perception and reasoning. |
| Approach: | They propose a family of vision-language models that have LeArned to Think wiTh vision spEcialists by offloading perception to state-of-the-art vision models. |
| Outcome: | The proposed model achieves 4-5% gains over baselines across 6 benchmarks covering both perception and reasoning abilities. |
Whodunit? Learning to Contrast for Authorship Attribution (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches to authorship attribution are dataset-dependent and yield inconsistent performance across corpora. |
| Approach: | They propose to fine-tune pre-trained generic language representations with a contrastive objective to learn author-specific representations by identifying clusters of authors. |
| Outcome: | The proposed approach improves on multiple human and machine authorship attribution benchmarks, but at the cost of sacrificing performance for some authors. |
GCRC: A New Challenging MRC Dataset from Gaokao Chinese for Explainable Evaluation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing machine reading comprehension datasets lack an explainable evaluation of systems' reasoning capabilities. |
| Approach: | They propose a dataset with multi-choice questions that evaluates MRC systems' reasoning process . they use sentence-level relevant supporting facts, error reason of distractors to evaluate MRC . |
| Outcome: | The proposed dataset is more challenging and useful for identifying limitations of existing MRC systems in an explainable way. |
Mitigating Reversal Curse in Large Language Models via Semantic-aware Permutation Training (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved impressive performance across diverse tasks, but suffer from the "reversal curse" this limitation poses a challenge to the advancement of artificial general intelligence (AGI) |
| Approach: | They propose to use training data to permute training sentences into entities and feed them into the model. |
| Outcome: | The proposed method improves the performance of large language models (LLMs) on reversed questions and improves existing models. |
Unleashing the Power of Emojis in Texts via Self-supervised Graph Pre-Training (2024.emnlp-main)
Copied to clipboard
| Challenge: | Emojis have gained immense popularity on social media platforms, serving as a common means to supplement or replace text. |
| Approach: | They propose a graph pre-train framework for text and emoji co-modeling that incorporates two tasks: node-level graph contrastive learning and edge-level link reconstruction learning. |
| Outcome: | The proposed framework improves on the Xiaohongshu and Twitter datasets with two types of downstream tasks. |
From Personal to Collective: On the Role of Local and Global Knowledge in LLM Personalization (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are the backbone of modern AI systems supporting a wide range of applications. |
| Approach: | They propose a Local–Global knowledge framework that augments user-specific signals with a global knowledge encoding collective behavioral trends. |
| Outcome: | The proposed framework outperforms existing methods on five personalization benchmarks. |
Can LLM Graph Reasoning Generalize beyond Pattern Memorization? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies seek to enhance the graph reasoning capabilities of Large Language Models (LLMs) by specialized instruction tuning. |
| Approach: | They propose to evaluate LLM graph reasoning generalization using in-distribution settings . they propose to use three strategies to improve LLM generalization . |
| Outcome: | The proposed benchmark evaluates LLM graph reasoning generalization with in-distribution settings only . it shows that LLMs struggle to generalize across reasoning and real-world patterns . |
Learned Adapters Are Better Than Manually Designed Adapters (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve adapter-based tuning are sub-optimal . a learning framework is proposed to learn the optimal adapter architectures . |
| Approach: | They propose a framework to automatically learn optimal adapter architectures for better task adaptation of pre-trained models. |
| Outcome: | The proposed framework outperforms the previous parameter-efficient tuning baselines while tuning comparable or fewer parameters. |
GroupRank: A Groupwise Paradigm for Effective and Efficient Passage Reranking with LLMs (2026.findings-acl)
Copied to clipboard
Meixiu Long, Duolin Sun, Dan Yang, Yihan Jiao, Lei Liu, Jiahai Wang, Binbin Hu, Yue Shen, Jie Feng, Zhehao Tan, Junjie Wang, Lianzhen Zhong, Jian Wang, Peng Wei, Jinjie Gu
| Challenge: | Existing rankers excel in lexical-matching scenarios, while they struggle with complex queries requiring deep reasoning. |
| Approach: | They propose a new paradigm that balances flexibility and context awareness to unlock the full potential of groupwise reranking. |
| Outcome: | The proposed approach achieves a state-of-the-art 65.2 NDCG@10 on BRIGHT and surpasses baselines by 2.1 points on R2MED while delivering a 6.4 inference speedup. |
daDPO: Distribution-Aware DPO for Distilling Conversational Abilities (2025.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) with Direct Preference Optimization (DPO) has emerged as a promising approach to enhance the conversational abilities of smaller models using a larger teacher model. |
| Approach: | They propose a framework that integrates the teacher's distributional information into DPO distillation while preserving theoretical guarantees. |
| Outcome: | The proposed framework outperforms existing methods in restoring performance for pruned models and enhancing smaller models within the same LLM family. |
NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation Tasks (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Graphical User Interface (GUI) and embodied navigation have driven progress, yet these domains have largely evolved in isolation, with disparate datasets and training paradigms. |
| Approach: | They propose a visual-target trajectory collection pipeline that generates trajectories for GUI and embodied tasks using a single formulation. |
| Outcome: | The proposed agent outperforms state-of-the-art agents in GUI navigation, spatial affordance prediction, and embodied navigation. |
FRVA: Fact-Retrieval and Verification Augmented Entailment Tree Generation for Explainable Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for generating a entailment tree exhibit the reasoning chains from knowledge facts to predicted answers, but they have large fact search spaces and error accumulation problems resulting in the generation of invalid steps. |
| Approach: | They propose a Fact-Retrieval and Verification Augmented bidirectional entailment tree generation method that contains two systems. |
| Outcome: | The proposed method outperforms existing models and achieves state-of-the-art performance in fact selection and structural correctness. |
Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition (2025.acl-long)
Copied to clipboard
Kehua Feng, Keyan Ding, Tan Hongzhi, Kede Ma, Zhihua Wang, Shuangquan Guo, Cheng Yuzhou, Ge Sun, Guozhou Zheng, Qiang Zhang, Huajun Chen
| Challenge: | Existing methods for evaluation of large language models are inefficient and inefficient due to inaccuracy of standard metrics in human perception of text quality and inefficiency in sampling informative test examples. |
| Approach: | They propose a sample-efficient human evaluation method for large language models based on the principle of MAximum Discrepancy (MAD) competition. |
| Outcome: | The proposed method achieves the “golden” ranking of LLMs with a minimum set of input instructions, which in turn reveal their relative strengths and weaknesses. |
ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models (2024.acl-long)
Copied to clipboard
Haochen Tan, Zhijiang Guo, Zhan Shi, Lu Xu, Zhili Liu, Yunlong Feng, Xiaoguang Li, Yasheng Wang, Lifeng Shang, Qun Liu, Linqi Song
| Challenge: | Existing evaluation methods for large language models are labor-intensive and lack efficiency. |
| Approach: | They propose a framework dedicated to assessing long-text generation that includes in-depth human-curated meta-questions spanning various domains . they use a set of proxy-quests with pre-annotated answers to assess the content's quality by incorporating the generated texts as contextual background. |
| Outcome: | The proposed framework assesses the quality of long-text content by matching it with references through human evaluation or automated metrics. |
An Efficient Conversational Smart Compose System (2023.acl-demo)
Copied to clipboard
Yun Zhu, Xiayu Chen, Lei Shu, Bowen Tan, Xinying Song, Lijuan Liu, Maria Wang, Jindong Chen, Ning Ruan
| Challenge: | a cloud-based smart compose system is designed to improve human-to-human conversation efficiency. |
| Approach: | They propose a cloud-based smart compose system to improve conversation efficiency . they propose heuristics to achieve the best trade-off between quality and latency . |
| Outcome: | The proposed system reduces latency without losing composing quality further. |
V-Oracle: Making Progressive Reasoning in Deciphering Oracle Bones for You and Me (2025.acl-long)
Copied to clipboard
Runqi Qiao, Qiuna Tan, Guanting Dong, MinhuiWu MinhuiWu, Jiapeng Wang, YiFan Zhang, Zhuoma GongQue, Chong Sun, Yida Xu, Yadong Xue, Ye Tian, Zhimin Bao, Lan Yang, Chen Li, Honggang Zhang
| Challenge: | Deciphering oracle bone scripts using AI technology is not an overnight task due to the evolution of written language over millennia. |
| Approach: | They propose a framework that utilizes Large Multi-modal Models (LMMs) for interpreting Oracle Bone Script (OBS). |
| Outcome: | The proposed framework provides quantitative analyses and superior deciphering capability. |
Digging out Discrimination Information from Generated Samples for Robust Visual Question Answering (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to solve this problem rely on additional annotations and generate negative samples . |
| Approach: | They propose a method to Dig out Discrimination information from Generated samples to address these limitations. |
| Outcome: | The proposed method improves on the visual question-answering datasets. |
SHARP: Steering Hallucination in LVLMs via Representation Engineering (2025.emnlp-main)
Copied to clipboard
Junfei Wu, Yue Ding, Guofan Liu, Tianze Xia, Ziyue Huang, Dianbo Sui, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan
| Challenge: | Large Vision-Language Models (LVLMs) generate responses that are plausible but incorrect or unsupported—commonly referred to as hallucinations. |
| Approach: | They propose a representation-level intervention framework that modulates hallucination-related features during inference by probing their encoded features. |
| Outcome: | The proposed framework reduces hallucinations while maintaining the performance and generalization capabilities of Large Vision-Language Models (LVLMs). |
Speculative Decoding for Multi-Sample Inference (2025.findings-emnlp)
Copied to clipboard
Yiwei Li, Jiayi Shi, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yueqi Zhang, Ji Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Speculative decoding method exploits consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases. |
| Approach: | They propose a speculative decoding method that exploits the consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or external databases. |
| Outcome: | The proposed method exploits the intrinsic consensus of parallel reasoning paths to synthesize high-quality draft tokens without auxiliary models or databases. |
DQ-BART: Efficient Sequence-to-Sequence Model via Joint Distillation and Quantization (2022.acl-short)
Copied to clipboard
Zheng Li, Zijian Wang, Ming Tan, Ramesh Nallapati, Parminder Bhatia, Andrew Arnold, Bing Xiang, Dan Roth
| Challenge: | Empirical analyses show that pre-trained sequence-to-sequence models can achieve a 16.5x model footprint compression ratio with little performance drop relative to full-precision counterparts. |
| Approach: | They propose to distill and quantize pre-trained sequence-to-sequence models to reduce memory and latency requirements. |
| Outcome: | Empirical results show that the proposed model achieves 16.5x model footprint compression ratio with little performance drop relative to full-precision counterparts on multiple summarization and QA datasets. |
SeaLLMs - Large Language Models for Southeast Asia (2024.acl-demos)
Copied to clipboard
Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Zhiqiang Hu, Chenhui Shen, Yew Ken Chia, Xingxuan Li, Jianyu Wang, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, Lidong Bing
| Challenge: | Existing large language models favor high-resource languages, such as English, at the expense of low-resourced and regional languages. |
| Approach: | They propose a series of language models that specifically focuses on Southeast Asian languages. |
| Outcome: | SeaLLM models outperform ChatGPT-3.5 in non-Latin languages by large margins . linguistic disparity impedes access to state-of-the-art AI technologies for non-English-speaking populations . |
On the Language Coverage Bias for Neural Machine Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | Language coverage bias is important for neural machine translation because of the target-original training data. |
| Approach: | They propose two approaches to alleviate the language coverage bias problem by explicitly distinguishing between the source-and target-original training data. |
| Outcome: | The proposed methods improve translation tasks on both back-and forward-translation and their tagged variants. |
BotSIM: An End-to-End Bot Simulation Framework for Commercial Task-Oriented Dialog Systems (2022.emnlp-demos)
Copied to clipboard
| Challenge: | evaluating and troubleshooting production TOD systems is still a largely manual process requiring large amount of human conversations with the systems. |
| Approach: | They propose a data-efficient end-to-end Bot SIMulation framework for commercial task-oriented dialog systems that can generate user queries and generate semantic-level dialog acts and entities from bot definitions. |
| Outcome: | The proposed framework is able to infer semantic-level dialog acts and entities from bot definitions and generate user queries via model-based paraphrasing. |
Tree-CoT-RT: An Explainable Multi-Path Tree-Guided Chain-of-Thought and Reinforcement Learning Framework for Aspect Sentiment Quad Prediction (2026.findings-acl)
Copied to clipboard
Hao Zhang, Jiahao Wang, Zhenke Duan, Xin Yin, Haichuan Hu, Hualong Chen, null Suyi, Congqing He, Yike Tan, Yu-N Cheah
| Challenge: | Existing methods lack explainability and generalization, making it difficult to justify inference decisions and detect implicit sentiment across domains and varied expression patterns. |
| Approach: | They propose an explainable multi-path tree-guided chain-of-thought framework specifically designed for ASQP. |
| Outcome: | Experiments on benchmark datasets show that Tree-CoT-RT outperforms baselines. |
CrystalICL: Enabling In-Context Learning for Crystal Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for crystal generation are limited to zero-shot scenarios and are unable to benefit from few-shot situations. |
| Approach: | They propose a model designed for few-shot crystal generation that exploits in-context learning by capturing structure-property relationships from limited data. |
| Outcome: | The proposed model reduces complexity of modeling crystal symmetry in LLMs and exploits ICL by capturing structure-property relationships from limited data. |
AnyMAC: Cascading Flexible Multi-Agent Collaboration via Next-Agent Prediction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for multi-agent collaboration rely on static or graph-based topologies lacking flexibility and adaptability. |
| Approach: | They propose a new framework that rethinks multi-agent coordination through a sequential structure rather than a graph structure. |
| Outcome: | The proposed method achieves superior performance while significantly reducing communication overhead. |
A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Non-autoregressive translation models suffer from the multi-modality problem when a source sentence corresponds to multiple correct translations. |
| Approach: | They propose to decompose the syntactic multi-modality problem into short- and long-range models and evaluate them on synthesized and real datasets. |
| Outcome: | The proposed loss functions can handle short- and long-range syntactic multi-modalities better than existing models. |
PAR: Political Actor Representation Learning with Social Context and Expert Knowledge (2022.emnlp-main)
Copied to clipboard
Shangbin Feng, Zhaoxuan Tan, Zilong Chen, Ningnan Wang, Peisheng Yu, Qinghua Zheng, Xiaojun Chang, Minnan Luo
| Challenge: | Existing approaches focus on textual data and voting records to induce political actors' stances. |
| Approach: | They propose a Political Actor Representation learning framework that leverages social context and expert knowledge to model ideological stances. |
| Outcome: | The proposed framework improves political text understanding and improves roll call vote prediction and political perspective detection. |
FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models (2025.emnlp-main)
Copied to clipboard
Hengxing Cai, Jinhan Dong, Jingjun Tan, Jingcheng Deng, Sihang Li, Zhifeng Gao, Haidong Wang, Zicheng Su, Agachai Sumalee, Renxin Zhong
| Challenge: | Existing methods for vision-and-language navigation struggle with insufficient multimodal fusion, weak generalization, and poor interpretability. |
| Approach: | They propose a framework for UAV vision-and-language navigation that integrates natural language instructions with visual observations to improve multimodal fusion and interpretability. |
| Outcome: | The proposed framework achieves state-of-the-art performance across all scenarios, with a 9.22% higher success rate than the strongest baseline in unseen environments. |
Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? (2025.findings-acl)
Copied to clipboard
| Challenge: | Analogical reasoning is a unique ability of humans to address unfamiliar challenges by transferring strategies from relevant past experiences. |
| Approach: | They propose to use self-generated random examples to improve performance on a variety of reasoning tasks by incorporating relevant examples from relevant past experiences. |
| Outcome: | The proposed methods achieve comparable or even better performance on GSM8K with random biological examples. |
ContraCLM: Contrastive Learning For Causal Language Model (2023.acl-long)
Copied to clipboard
Nihal Jain, Dejiao Zhang, Wasi Uddin Ahmad, Zijian Wang, Feng Nan, Xiaopeng Li, Ming Tan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Xiaofei Ma, Bing Xiang
| Challenge: | Existing studies show that causal language models lack expressiveness due to poor discrimination ability. |
| Approach: | They propose a contrastive learning framework that enhances discrimination of representations and bridges the gap with encoder-only models. |
| Outcome: | The proposed framework improves discrimination and source code generation capabilities on a variety of downstream tasks. |
RedCoast: A Lightweight Tool to Automate Distributed Training of LLMs on Any GPU/TPUs (2024.naacl-demo)
Copied to clipboard
| Challenge: | Recent advances in machine learning (ML) are attributed to large language models (LLMs), but their escalating memory requirements require developers to partition a large model to distribute it across multiple GPUs or TPUs. |
| Approach: | They propose a lightweight and user-friendly tool to automate distributed training and inference for LLMs and to simplify ML pipeline development. |
| Outcome: | The proposed tool automates distributed training and inference for LLMs, and simplifies ML pipeline development. |
REACT: Representation Extraction And Controllable Tuning to Overcome Overfitting in LLM Knowledge Editing (2025.emnlp-main)
Copied to clipboard
Haitian Zhong, Yuhuan Liu, Ziyang Xu, Guofan Liu, Qiang Liu, Shu Wu, Zhe Zhao, Liang Wang, Tieniu Tan
| Challenge: | Large language model editing methods suffer from overfitting, where factual updates can propagate beyond their intended scope, overemphasizing the edited target even when it’s contextually inappropriate. |
| Approach: | They propose a framework for precise and controllable knowledge editing that utilizes two-phase representations and a linear transformation to compute a directional "belief shift" vector. |
| Outcome: | The proposed framework significantly reduces overfitting across nearly all evaluation metrics and on COUNTERFACT and MQuAKE. |
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing knowledge distillation frameworks for language models are limited by memory and the use of complex distillation methods on larger-scale PLMs. |
| Approach: | They propose a general knowledge distillation framework that supports distillation on larger-scale PLMs using various distillation methods. |
| Outcome: | The proposed framework can support distillation on larger-scale PLMs and 25 mainstream methods on 8 NVIDIA A100 (40GB) GPUs. |
MAF: Multimodal Alignment Framework for Weakly-Supervised Phrase Grounding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on phrase localization uses caption-image datasets as weak supervision . existing work on supervised phrase localisation uses a large-scale annotated dataset . |
| Approach: | They develop a multimodal alignment framework to leverage more widely available caption-image datasets to model phrase relevance. |
| Outcome: | The proposed model improves on the widely-adopted Flickr30k dataset . it also improves the previous best unsupervised result by 5.56% . |
Parallel Instance Query Network for Named Entity Recognition (2022.acl-long)
Copied to clipboard
Yongliang Shen, Xiaobin Wang, Zeqi Tan, Guangwei Xu, Pengjun Xie, Fei Huang, Weiming Lu, Yueting Zhuang
| Challenge: | Named entity recognition is a fundamental task in natural language processing. |
| Approach: | They propose a method that sets up global and learnable instance queries to extract entities from a sentence in a parallel manner. |
| Outcome: | The proposed method outperforms existing state-of-the-art models on nested and flat datasets. |
From Sub-Ability Diagnosis to Human-Aligned Generation: Bridging the Gap for Text Length Control via MarkerGen (2025.acl-long)
Copied to clipboard
Peiwen Yuan, Chuyi Tan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Jiayi Shi, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Existing methods to control text length are lacking in LCTG, posing a major limitation for practical applications. |
| Approach: | They propose a plug-and-play approach that decomposes LCTG sub-abilities with human patterns as reference and performs detailed error analysis. |
| Outcome: | The proposed method significantly improves LCTG across various settings, exhibiting outstanding effectiveness and generalizability. |
Chinese SimpleQA: A Chinese Factuality Evaluation for Large Language Models (2025.acl-long)
Copied to clipboard
Yancheng He, Shilong Li, Jiaheng Liu, Yingshui Tan, Weixun Wang, Hui Huang, Xingyuan Bu, Hangyu Guo, Chengwei Hu, Boren Zheng, Zhuoran Lin, Dekai Sun, Zhicheng Zheng, Wenbo Su, Bo Zheng
| Challenge: | Current frontier models sometimes generate false outputs or answers that are not substantiated by evidence. |
| Approach: | They propose Chinese SimpleQA, a Chinese benchmark to evaluate LLMs' factuality . they focus on Chinese language over 6 major topics with 99 diverse subtopics . |
| Outcome: | The Chinese SimpleQA benchmark evaluates the factuality ability of LLMs . the questions and answers are short and easy-to-evaluate . |
Semantic-Eval : A Semantic Comprehension Evaluation Framework for Large Language Models Generation without Training (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have emerged as key drivers of progress in the field of natural language processing. |
| Approach: | They propose a framework that assesses LLM-generated text based on semantic understanding. |
| Outcome: | The proposed framework surpasses traditional evaluation metrics and lags behind GPT-4. |
HydraRAG: Structured Cross-Source Enhanced Large Language Model Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current RAG system retrieves evidence from knowledge graphs and text documents but has limitations in multi-hop reasoning, multi-entity questions, and source verification. |
| Approach: | They propose a training-free framework that unifies graph topology, document semantics, and source reliability to support deep, faithful reasoning in large language models. |
| Outcome: | The proposed framework outperforms the current hybrid model-based model-driven system by 20.3% and 30.1% on seven benchmark datasets. |
XtremeCLIP: Extremely Parameter-efficient Tuning for Low-resource Vision Language Understanding (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune visual-language understanding (VLU) require tasks-specific designs and sufficient training data. |
| Approach: | They propose a simple yet efficient paradigm for low-resource Visual Language Understanding (VLU) they reformulate a series of VLU tasks as an open-book affinity-matching problem. |
| Outcome: | The proposed framework outperforms baselines in low-resource settings. |
Generalizable LLM Learning of Graph Synthetic Data with Post-training Alignment (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing research has focused on enhancing graph reasoning capabilities of LLMs by supervised fine-tuning on synthetic graph data. |
| Approach: | They propose to unlock generalizable learning of graph with post-training alignment with synthetic graph data by aligning off-the-shelf LLMs and LLM fine-tuned on synthetic graphs. |
| Outcome: | The proposed algorithm improves on synthetic graph problems and out-of-domain tasks with implicit graph structures. |
Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation (2025.findings-acl)
Copied to clipboard
Yiwei Li, Ji Zhang, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Jiayi Shi, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Existing studies on self-consistency show that it improves reasoning abilities by aggregating diverse stochastic samples. |
| Approach: | They propose a confidence-driven mechanism that dynamically calibrates temperature to align with high probability modes. |
| Outcome: | The proposed method outperforms fixed-diversity baselines on reasoning tasks and improves both average and best-case performance. |
Surprise Calibration for Better In-Context Learning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for in-context learning apply fixed class priors across all inputs . existing methods rely on retraining and retrain models . |
| Approach: | They propose a Bayesian-based method to capture the temporal dynamics of class priors . they identify "surprise" as an informative signal for class prior shift . |
| Outcome: | The proposed method outperforms existing methods on a range of benchmark tasks. |
Neural Machine Translation with Decoding History Enhanced Attention (C18-1)
Copied to clipboard
| Challenge: | Neural machine translation with source-side attention has been criticized for its poor memory performance. |
| Approach: | They propose to use a Decoding History Enhanced Attention mechanism to render NMT models better at selecting both source-side and target-side information. |
| Outcome: | The proposed model improves by 0:9 BLEU on Chinese-English translation and the state-of-the-art on a larger task. |
MedCite: Can Language Models Generate Verifiable Text for Medicine? (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing LLM-based medical question answering systems lack citation generation and evaluation capabilities, raising concerns about their adoption in practice. |
| Approach: | They propose a framework that facilitates the design and evaluation of LLM citations for medical tasks and a retrieval-citation method that generates high-quality citation. |
| Outcome: | The proposed method achieves superior citation precision and recall improvements compared to strong baseline methods and correlates well with annotation results from professional experts. |
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training (2021.findings-acl)
Copied to clipboard
| Challenge: | Symbolic music understanding is useful for many music applications, but lack of training data hinders representation learning. |
| Approach: | They propose a pre-trained model for music understanding that uses symbolic music data to train music representations. |
| Outcome: | The proposed model improves on four music understanding tasks. |
CompTab: A Comprehensive Benchmark for Real-World TableQA with Complex Reasoning and Irregular Tables (2026.acl-long)
Copied to clipboard
Zhen Yang, Wei Du, Jie Wang, Wenze Zhou, Xiangfeng Meng, Zhengyang Wang, Suping Sun, Ziwei Du, Haodong Zou, Jie Chen, Yongbin Liu, Shicheng Tan, Jiahao Ying, Shu Zhao
| Challenge: | Existing benchmarks focus on well-structured tables and fail to reflect irregular structures and complex reasoning commonly encountered in real-world scenarios. |
| Approach: | They propose a benchmark to evaluate TableQA under complex reasoning and irregular table conditions. |
| Outcome: | The proposed framework improves generalization and realism of large language models under complex and irregular table conditions. |
xLAM: A Family of Large Action Models to Empower AI Agent Systems (2025.naacl-long)
Copied to clipboard
Jianguo Zhang, Tian Lan, Ming Zhu, Zuxin Liu, Thai Quoc Hoang, Shirley Kokane, Weiran Yao, Juntao Tan, Akshara Prabhakar, Haolin Chen, Zhiwei Liu, Yihao Feng, Tulika Manoj Awalgaonkar, Rithesh R N, Zeyuan Chen, Ran Xu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Silvio Savarese, Caiming Xiong
| Challenge: | Autonomous agents powered by large language models (LLMs) have attracted significant research interest, but there are few standards for developing specialized models for agent tasks. |
| Approach: | They propose a series of large action models with dense and mixture-of-expert architectures that unifies, augments, and synthesizes diverse datasets to enhance agent generalizability and performance. |
| Outcome: | The proposed models outperform GPT-4, Claude-3, and many other models in terms of tool use and outperformed GPT-based models on multiple agent ability benchmarks. |
Guess What I am Thinking: A Benchmark for Inner Thought Reasoning of Role-Playing Language Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs)-based Role-Playing Language Agents (RPLAs) have attracted broad attention in various applications. |
| Approach: | They propose a benchmark for evaluating character thought generation using literature . they propose 'MIRROR' which generates character thoughts by retrieving memories, predicting character reactions, and synthesizing motivations. |
| Outcome: | The proposed benchmark outperforms existing methods in evaluating character thought generation. |
Large Language Models for Data Annotation and Synthesis: A Survey (2024.emnlp-main)
Copied to clipboard
Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, Huan Liu
| Challenge: | Existing surveys focus on LLMs' specific utility for data annotation and synthesis. |
| Approach: | They propose to use large language models to generate annotations from raw data . they also propose to review learning strategies for models utilizing LLM-generated annotations . |
| Outcome: | The proposed models can be used to improve the efficacy of machine learning models by generating and labeling raw data with relevant information. |
UnifiedVisual: A Framework for Constructing Unified Vision-Language Datasets (2025.emnlp-main)
Copied to clipboard
Pengyu Wang, Shaojun Zhou, Chenkun Tan, Xinghao Wang, Wei Huang, Zhen Ye, Zhaowei Li, Botian Jiang, Dong Zhang, Xipeng Qiu
| Challenge: | Existing datasets address understanding and generation in isolation, limiting the performance of unified vision large language models. |
| Approach: | They propose a dataset that facilitates mutual enhancement between multimodal understanding and generation. |
| Outcome: | The proposed framework integrates diverse visual and textual inputs and outputs, enabling comprehensive cross-modal reasoning and precise text-to-image alignment. |
Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a well-studied task in natural language processing. |
| Approach: | They propose a method that generates span proposals and labels them with categories . they use boundary information of entities and partially matched spans to locate them . |
| Outcome: | The proposed method outperforms state-of-the-art models on nested NER datasets. |
Guiding Dialogue Agents to Complex Semantic Targets by Dynamically Completing Knowledge Graph (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing knowledge graphs are incomplete in tracking complex semantic relations of the target-oriented dialogue. |
| Approach: | They combine methods of knowledge retrieval and relationship prediction to construct a context-related dynamic KG and a metric to evaluate the tracked path automatically. |
| Outcome: | The proposed method can control the agent more logically and smoothly toward the complex target. |
CAML: A Conflict-Aware Molecular Language Model Merging Framework for Multi-Constraint Molecular Generation (2026.acl-long)
Copied to clipboard
Xuanbai Ren, Luoda Tan, Pei Liu, Tengfei Ma, Xiangzheng Fu, Longyue Wang, Yiping Liu, Xiangxiang Zeng
| Challenge: | Existing paradigms struggle with this challenge due to catastrophic forgetting or gradient conflicts. |
| Approach: | They propose a conflict-aware molecular language model merging framework that generates multiple constraints moleculaire as a cooperative game among property-specific fine-tune models. |
| Outcome: | The proposed framework generates multiple constraints molecular as a cooperative game among property-specific fine-tune models (expert models) it minimizes conflicts among properties by exploring the optimal combination of the importance of the task parameter and relative fusion weights of each expert (fusion coefficient). |
Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features (N19-2)
Copied to clipboard
Wei Yang, Luchen Tan, Chunwei Lu, Anqi Cui, Han Li, Xi Chen, Kun Xiong, Muzi Wang, Ming Li, Jian Pei, Jimmy Lin
| Challenge: | e-commerce companies often have the option of escalating complaints by filing grievances with a government authority . this is detrimental to an ecommerce company, but this problem is challenging to solve by integrating recurrent neural networks with manually-engineered features. |
| Approach: | They propose a model that integrates recurrent neural networks with manually-engineered features to identify cases where the customer expresses such an intent. |
| Outcome: | The proposed model outperforms baseline models and provides better recall and triage for specialized agents. |
M2RC-EVAL: Massively Multilingual Repository-level Code Completion Evaluation (2025.acl-long)
Copied to clipboard
Jiaheng Liu, Ken Deng, Congnan Liu, Jian Yang, Shukai Liu, He Zhu, Peng Zhao, Linzheng Chai, Yanan Wu, JinKe JinKe, Ge Zhang, Zekun Moore Wang, Guoan Zhang, Yingshui Tan, Bangyu Xiang, Zhaoxiang Zhang, Wenbo Su, Bo Zheng
| Challenge: | Existing repository-level code completion benchmarks focus on a limited number of languages . existing benchmarks report overall average scores of different languages ignoring fine-grained abilities . |
| Approach: | They propose to use repository-level code completion benchmarks to evaluate general code intelligence abilities across languages for existing code Large Language Models. |
| Outcome: | The proposed benchmarks improve the code completion abilities of existing LLMs by using two types of annotations on the parsed syntax tree. |
LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)
Copied to clipboard
Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
A Template-based Method for Constrained Neural Machine Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to solve this problem can not satisfy the following three desiderata: (1) high translation quality, (2) high match accuracy, and (3) low latency. |
| Approach: | They propose a template-based method that can provide high translation quality and match accuracy and a low latency inference. |
| Outcome: | The proposed method outperforms baselines in lexically and structurally constrained translation tasks and can be used in a variety of applications. |
BaseCal: Unsupervised Confidence Calibration via Base Model Signals (2026.acl-long)
Copied to clipboard
Hexiang Tan, Wanli Yang, Junwei Zhang, Xin Chen, Rui Tang, Du Su, Jingang Wang, Yuanzhuo Wang, Fei Sun, Xueqi Cheng
| Challenge: | Post-trained LLMs typically compromise reliability with severe overconfidence, resulting in inaccurate responses. |
| Approach: | They propose a solution that feeds PoLLMs into the base LLM to get confidence. |
| Outcome: | The proposed solution reduces expected calibration error (ECE) by 42.90% compared to the best unsupervised baselines. |
Separate the Wheat from the Chaff: Winnowing Down Divergent Views in Retrieval Augmented Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) lack robustness in knowledge-intensive tasks due to noisy or irrelevant retrieved data. |
| Approach: | They propose a multi-agent debate-based RAG framework that integrates external knowledge sources into large language models to improve their accuracy. |
| Outcome: | The proposed framework is unsupervised and leverages pretrained LLMs without fine-tuning, making it easily adaptable to various tasks. |
VenusFactory: An Integrated System for Protein Engineering with Data Retrieval and Language Model Fine-Tuning (2025.acl-demo)
Copied to clipboard
Yang Tan, Chen Liu, Jingyuan Gao, Wu Banghao, Mingchen Li, Ruilin Wang, Lingrong Zhang, Huiqun Yu, Guisheng Fan, Liang Hong, Bingxin Zhou
| Challenge: | Pre-trained protein language models have been used in protein engineering, but their adoption is limited due to data collection, task benchmarking, and application challenges. |
| Approach: | They propose a versatile engine that integrates biological data retrieval, standardized task benchmarking, and modular fine-tuning of PLMs. |
| Outcome: | The proposed engine integrates biological data retrieval, task benchmarking, and modular fine-tuning of PLMs. |
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing (2021.acl-demo)
Copied to clipboard
Xiao Wang, Qin Liu, Tao Gui, Qi Zhang, Yicheng Zou, Xin Zhou, Jiacheng Ye, Yongxin Zhang, Rui Zheng, Zexiong Pang, Qinzhuo Wu, Zhengyan Li, Chong Zhang, Ruotian Ma, Zichu Fei, Ruijian Cai, Jun Zhao, Xingwu Hu, Zhiheng Yan, Yiding Tan, Yuan Hu, Qiyuan Bian, Zhihua Liu, Shan Qin, Bolin Zhu, Xiaoyu Xing, Jinlan Fu, Yue Zhang, Minlong Peng, Xiaoqing Zheng, Yaqian Zhou, Zhongyu Wei, Xipeng Qiu, Xuanjing Huang
| Challenge: | Existing approaches to textual robustness evaluation focus on slightly modifying the input data, which maintains the original meaning and results in a different prediction. |
| Approach: | They propose a multilingual robustness evaluation toolkit for NLP that integrates universal text transformations, task-specific transformations and adversarial attack. |
| Outcome: | The toolkit includes universal text transformation, task-specific transformation, adversarial attack, subpopulation, and their combinations to provide comprehensive robustness analyses. |
Extract and Attend: Improving Entity Translation in Neural Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to improve entity translation in Neural machine translation still suffer from inaccurate translation of entities due to the lack of entity training instances. |
| Approach: | They propose an extract-and-tend approach to enhance entity translation in NMT by extracting entities from a dictionary and attending to them with a prefix. |
| Outcome: | Experiments on En-Zh and En-Ru show that the proposed approach improves translation accuracy and translation quality. |
Latent-Condensed Transformer for Efficient Long Context Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches address these bottlenecks separately: Multi-head Latent Attention (MLA) reduces the KV cache by projecting tokens into a low-dimensional latent space, while sparse attention reduces computation. |
| Approach: | They propose a Latent-Condensed Attention mechanism that performs structured context condensation directly within MLA's latent space. |
| Outcome: | The proposed approach reduces KV cache size and attention cost without adding parameters. |
Closing the Loop: Learning to Generate Writing Feedback via Language Model Simulated Student Revisions (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in language models (LMs) have made it possible to automatically generate feedback that is actionable and well-aligned with human-specified attributes. |
| Approach: | They propose a tool that PROduces Feedback via learning from LM simulated student revisions and propose to iteratively optimize the feedback generator by directly maximizing the effectiveness of students’ overall revising performance. |
| Outcome: | The proposed approach surpasses baseline methods in effectiveness of improving students’ writing and demonstrates enhanced pedagogical values, even though it was not explicitly trained for this aspect. |
TiMem: Temporal-Hierarchical Memory Consolidation for Long-Horizon Conversational Agents (2026.findings-acl)
Copied to clipboard
Kai Li, Xuanqing Yu, Ziyi Ni, Yi Zeng, Yao Xu, Zheqing Zhang, Xin Li, Jitao Sang, Xiaogang Duan, Xuelei Wang, Chengbao Liu, Jie Tan
| Challenge: | Existing memory frameworks provide limited support for temporally structured information across hierarchical levels, leading to fragmented memories and unstable long-horizon personalization. |
| Approach: | They propose a temporal–hierarchical memory framework that organizes conversations through a Temporal Memory Tree. |
| Outcome: | The proposed framework outperforms baselines while reducing the recalled memory length by 52.20%. |
SlackAgents: Scalable Collaboration of AI Agents in Workspaces (2025.emnlp-demos)
Copied to clipboard
Zhiwei Liu, Weiran Yao, Zuxin Liu, Juntao Tan, Jianguo Zhang, Frank Wang, Sukhandeep Nahal, Huan Wang, Shelby Heinecke, Silvio Savarese, Caiming Xiong
| Challenge: | Existing open-source frameworks like LangChain and LlamaIndex fail to integrate into daily workflows, resulting in limited daily usage for work. |
| Approach: | They propose a multi-agent library for scalable management and collaboration of AI agents on Slack. |
| Outcome: | The proposed framework offers instant AI integration into organizational workflows and facilitates scalable collaboration, allowing for effective communication and task orchestration. |
Breaking the Impasse: Dual-Scale Evolutionary Policy Training for Social Language Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Reinforcement Learning with Verifiable Rewards (RLVR) is effective for closed-ended tasks, but it is not applicable to open-ended social language games. |
| Approach: | They propose a method that uses a time-scaled evolutionary perception mechanism to detect impasse by quantifying dual-scale value baseline divergence alongside match entropy. |
| Outcome: | Experiments on multiple social language games show that the proposed method outperforms baselines and avoids policy degeneration. |
Understanding the Information Propagation Effects of Communication Topologies in LLM-based Multi-Agent Systems (2025.emnlp-main)
Copied to clipboard
| Challenge: | Empirical studies for communication topology design often overlook why and when sparse and dense topologies help or hinder collaboration. |
| Approach: | They propose a topology design approach that balances error suppression and beneficial information propagation by fusing connectivity patterns from dense and sparse graphs. |
| Outcome: | The proposed topology design achieves superior performance across tasks with sparse and dense graphs. |
Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model (2024.emnlp-main)
Copied to clipboard
Wenqi Zhang, Zhenglin Cheng, Yuanyu He, Mengna Wang, Yongliang Shen, Zeqi Tan, Guiyang Hou, Mingqian He, Yanna Ma, Weiming Lu, Yueting Zhuang
| Challenge: | Using large language models, large multimodal models struggle with basic tasks like reading time from a clock and planning a route using a road map. |
| Approach: | They propose a multimodal self-instruct that synthesizes massive abstract images and visual reasoning instructions. |
| Outcome: | The proposed model synthesizes 11,193 abstract images and reasoning instructions across eight visual scenarios. |