Papers by Chen Gao
Exploiting domain-slot related keywords description for Few-Shot Cross-Domain Dialogue State Tracking (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing frameworks for dialogue state tracking with domain-slot-value labels are expensive . current models are limited due to high cost of data annotation and lack of data in some domains . |
| Approach: | They propose a framework based on domain-slot related description to tackle the challenge of few-shot cross-domain DST. |
| Outcome: | The proposed framework outperforms existing methods on MultiWOZ and gains strong slot accuracy compared to existing models. |
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
Capability Salience Vector: Fine-grained Alignment of Loss and Capabilities for Downstream Task Scaling Law (2025.acl-long)
Copied to clipboard
Qiming Ge, Shuhao Xing, Songyang Gao, Yunhua Zhou, Yicheng Zou, Songyang Zhang, Zhi Chen, Hang Yan, Qi Zhang, Qipeng Guo, Kai Chen
| Challenge: | Large language models have demonstrated impressive performance across a wide range of tasks, but this achievement comes with the trade-off of significant computational demands. |
| Approach: | They propose a scaling law that decomposes the overall validation loss and assigns different importance weights to tokens to assess a specific meta-capability. |
| Outcome: | The proposed model can predict the loss trending of models across different levels of computation without a gap between validation loss and model's downstream capabilities. |
Rethinking Word Similarity: Semantic Similarity through Classification Confusion (2025.naacl-long)
Copied to clipboard
| Challenge: | Word similarity measures cannot capture context-dependent, asymmetrical, polysemous nature of semantic similarity. |
| Approach: | They propose a new measure of similarity that reframes semantic similarity in terms of feature-based classification confusion. |
| Outcome: | The proposed model is comparable to cosine similarity in matching human similarity judgments across several datasets and can measure similarity using predetermined features of interest. |
Topology Imbalance and Relation Inauthenticity Aware Hierarchical Graph Attention Networks for Fake News Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods to detect fake news focus on mining lexical and syntactic features. |
| Approach: | They propose a topology imbalance and Relation inauthenticity aware Hierarchical Graph Attention Networks to identify fake news on social media. |
| Outcome: | The proposed method outperforms state-of-the-art methods on real-world datasets. |
Masked Thought: Simply Masking Partial Reasoning Steps Can Improve Mathematical Reasoning Learning of Language Models (2024.acl-long)
Copied to clipboard
Changyu Chen, Xiting Wang, Ting-En Lin, Ang Lv, Yuchuan Wu, Xin Gao, Ji-Rong Wen, Rui Yan, Yongbin Li
| Challenge: | Despite the advances in large language models, they still face difficulties with multi-step reasoning tasks. |
| Approach: | They propose a method that randomly masks certain tokens within the chain of thought to improve model accuracy by 5% over standard supervised fine-tuning. |
| Outcome: | The proposed method improves accuracy and accuracy by 5% over standard fine-tuning with a few codes modified. |
The Elephant in the Room: Exploring the Role of Neutral Words in Language Model Group-Agnostic Debiasing (2025.findings-acl)
Copied to clipboard
Xinwei Guo, Jiashi Gao, Junlei Zhou, Jiaxin Zhang, Guanhua Chen, Xiangyu Zhao, Quanying Liu, Haiyan Wu, Xin Yao, Xuetao Wei
| Challenge: | Large Language Models (LLMs) are increasingly integrated into our daily lives, raising ethical concerns, especially about perpetuating stereotypes. |
| Approach: | They propose a method that incorporates a neutral word semantics-based loss function to alleviate the deterioration of the LMS during debiasing. |
| Outcome: | The proposed method alleviates the deterioration of the Language Modeling Score (LMS) by incorporating a neutral word semantics-based loss function. |
SmartBench: Is Your LLM Truly a Good Chinese Smartphone Assistant? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks for Large Language Models focus on objective tasks like mathematics and coding in English, which do not reflect the practical use cases of on-device LLMs in real-world mobile scenarios. |
| Approach: | They propose a benchmark to evaluate the capabilities of on-device Large Language Models in Chinese mobile contexts. |
| Outcome: | The proposed framework evaluates on-device LLMs and MLLMs in Chinese . it provides a standardized framework for evaluating LLM performance on real smartphones . |
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness (2026.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) have shown promise in MER, but their internal decision-making mechanisms under modality conflict and missingness remain underexplored. |
| Approach: | They propose a multimodal large language model that can detect and control modality conflicts and missing subsets by a lightweight mechanism that detects and controls modality conflict. |
| Outcome: | The proposed framework improves performance across settings, showing it can handle conflict and missing behaviors. |
Large Language Models are Limited in Out-of-Context Knowledge Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) possess extensive knowledge and strong capabilities in performing in-context reasoning. |
| Approach: | They evaluated a dataset with seven representative OCKR tasks to assess their OCKr capabilities. |
| Outcome: | The model's OCKR abilities are limited regardless of whether the knowledge is trained in a separate or adjacent training setting. |
Understanding the Difficulty of Training Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Admin (Adaptive model initialization) is more stable, converges faster, and leads to better performance. |
| Approach: | They propose a model initialization algorithm to stabilize early training and unleash its full potential in the late stage. |
| Outcome: | The proposed model initialization method stabilizes early training and unleashes full potential in late stage. |
Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors (2025.emnlp-main)
Copied to clipboard
Hao Fang, Jiawei Kong, Tianqu Zhuang, Yixiang Qiu, Kuofeng Gao, Bin Chen, Shu-Tao Xia, Yaowei Wang, Min Zhang
| Challenge: | Existing methods to detect large language models (LLMs) generated for plagiarism use paraphrases to rewrite them to evade detection. |
| Approach: | They propose a training-free method that effectively fools text detectors using off-the-shelf LLMs by rewriting them to evade detection. |
| Outcome: | The proposed method deceives text detectors using off-the-shelf LLMs by rewriting them to produce human-like sentences that are less discernible by detectors. |
Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing evaluation methods do not provide insight into how well a language model captures distinct linguistic skills essential for language understanding and reasoning. |
| Approach: | They propose a new format of NLI benchmark for evaluation of broad-coverage linguistic phenomena using a set of datasets and an evaluation procedure for diagnosing how well a language model captures reasoning skills. |
| Outcome: | The proposed model can diagnose model behavior and verify model learning quality. |
Depression Detection on Social Media with Large Language Models (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for analyzing social media data lack a systematic integration of medical knowledge, causing a critical treatment gap. |
| Approach: | They propose a framework that leverages Large Language Models to integrate medical knowledge into social media data. |
| Outcome: | The proposed framework can be used to distinguish depression from transient mood changes. |
From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework (2023.findings-acl)
Copied to clipboard
Yangyi Chen, Hongcheng Gao, Ganqu Cui, Lifan Yuan, Dehan Kong, Hanlu Wu, Ning Shi, Bo Yuan, Longtao Huang, Hui Xue, Zhiyuan Liu, Maosong Sun, Heng Ji
| Challenge: | Existing models of robustness evaluation are incomprehensive, impractical, and invalid . |
| Approach: | They propose a framework for automatic robustness evaluation that shifts towards model-centric evaluation to further exploit the advantages of adversarial attacks. |
| Outcome: | The proposed framework is based on a model-centric evaluation protocol and a robustness evaluation protocol. |
Evaluating and Enhancing the Robustness of Code Pre-trained Models through Structure-Aware Adversarial Samples Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained code models have made significant strides in the field of neural code intelligence, but they are susceptible to adversarial attacks that subtly modify the input sequence and can impair generalization. |
| Approach: | They propose a set of novel robustness evaluation methods based on the intrinsic structure of the code to explore the impact of imperceptible perturbation. |
| Outcome: | The proposed methods have demonstrated their effectiveness across a wide range of models and tasks, and are able to predict the performance of perturbed models. |
AgentGym2: Benchmarking Large Language Model Agents in De-Idealized Real-World Environments (2026.acl-long)
Copied to clipboard
Zhiheng Xi, Dingwen Yang, Jiaqi Liu, Jixuan Huang, Honglin Guo, Baodai Huang, Tinggang Chen, Qi Zhang, Zhonghang Lu, Chenyu Liu, Jiajun Sun, Jiazheng Zhang, Dingwei Zhu, Xin Guo, Junzhe Wang, Zhihao Zhang, Yuming Yang, Junjie Ye, Minghe Gao, Dongrui Liu, Jiaming Ji, Guohao Li, Tao Gui, Qi Zhang, Xuanjing Huang
| Challenge: | Existing benchmarks evaluate agents in simplified, idealized settings, relying on pre-packaged tool interfaces, overlooking critical steps, and assume inputs are clean and fully specified. |
| Approach: | They propose a framework that evaluates language agents in simplified, idealized settings . they show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
| Outcome: | Experiments on 15 proprietary and open-source models show that even SOTA systems like Gemini and GPT-5 struggle on AgentGym2 . |
Towards Consistent Natural-Language Explanations via Explanation-Consistency Finetuning (2025.coling-main)
Copied to clipboard
| Challenge: | Large language models generate convincing, fluent explanations, but they often generate inconsistent explanations on different inputs. |
| Approach: | They propose a method that adapts large language models to generate more consistent explanations on related examples. |
| Outcome: | The proposed method yields a 10.0% relative explanation consistency improvement across a variety of question-answering datasets and generalizes to 7 out-of-distribution datasets not seen during finetuning (+4.5%) |
IMTLab: An Open-Source Platform for Building, Evaluating, and Diagnosing Interactive Machine Translation Systems (2023.emnlp-main)
Copied to clipboard
Xu Huang, Zhirui Zhang, Ruize Gao, Yichao Du, Lemao Liu, Guoping Huang, Shuming Shi, Jiajun Chen, Shujian Huang
| Challenge: | Existing systems that use a left-to-right completion paradigm are inefficient and expensive. |
| Approach: | They propose an open-source end-to-end interactive machine translation system platform . they propose to use a prefix-constrained decoding approach to achieve end- to-end evaluation . |
| Outcome: | The proposed system can guarantee high-quality, error-free translations . it uses prefix-constrained decoding and improves on previous systems . |
Make Prompt-based Black-Box Tuning Colorful: Boosting Model Generalization from Three Orthogonal Perspectives (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown increasing power on NLP tasks. however, tuning these models for downstream tasks usually requires exorbitant costs. |
| Approach: | They propose a black-box tuning technique that optimizes task-specific prompts without accessing gradients and hidden representations. |
| Outcome: | The proposed method improves performance under few-shot learning scenarios. |
DRBO: Mitigating Short Board Effect via Dynamic Reward Balancing in Multi-reward LLM Optimization (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework to optimize large language models (LLMs) for evaluation metrics is needed to balance weaker metrics. |
| Approach: | They propose a Dynamic Reward Balancing Optimization framework to mitigate the "short-board effect" they apply it to single-task and multi-type task scenarios . |
| Outcome: | The proposed framework improves performance and balances performance across multiple metrics. |
MED-COREASONER: Reducing Language Disparities in Medical Reasoning via Language-Informed Co-Reasoning (2026.acl-long)
Copied to clipboard
Fan Gao, Sherry T. Tong, Jiwoong Sohn, Jiahao Huang, Junfeng Jiang, Ding Xia, Piyalitt Ittichaiwong, Kanyakorn Veerakanjana, Hyunjae Kim, Qingyu Chen, Edison Marrese-Taylor, Kazuma Kobayashi, Akiko Aizawa, Irene Li
| Challenge: | Existing models that use English and local languages have a multilingual gap . a language-informed co-reasoning framework can be used to improve multilingual reasoning . |
| Approach: | They propose a language-informed co-reasoning framework that elicits parallel English and local-language reasoning and abstracts them into structured concepts. |
| Outcome: | Experiments show that Med-CoReasoner improves multilingual reasoning performance by 5% . the framework produces clinically sound and culturally grounded reasoning traces . |
Beyond Surface-Level Pattern Trap: LLM Agents for Faster and Smarter Cross-Architecture Code Migration (2026.findings-acl)
Copied to clipboard
| Challenge: | cross-architecture code migration is a resource-intensive and errorprone task. |
| Approach: | a framework for cross-architecture code migration is proposed to decouple implementation details through functional mining and code refactoring. |
| Outcome: | a new framework improves performance and correctness over state-of-the-art frameworks on OpenCV migration tasks. |
Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing related work generation models are inflexible and extract sentences from multiple papers to form a related work discussion. |
| Approach: | They propose a Relation-aware Related work generator which generates an abstractive related work from the given multiple scientific papers in the same research area. |
| Outcome: | The proposed model improves over existing models and can be used to familiarize researchers with the state of the art in the field. |
SocialBench: Sociality Evaluation of Role-Playing Conversational Agents (2024.findings-acl)
Copied to clipboard
Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Gao Xing, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, Fei Huang
| Challenge: | Existing studies on role-playing agents have focused on enhancing their conversational capability, role-specific knowledge and style, but there has been a gap in assessing their social intelligence. |
| Approach: | They propose a benchmark to evaluate the sociality of role-playing agents using LLMs. |
| Outcome: | The proposed benchmark is constructed from various sources and covers a wide range of 500 characters and over 6,000 question prompts and 30,800 multi-turn role-playing utterances. |
Reasoning in a Combinatorial and Constrained World: Benchmarking LLMs on Natural-Language Combinatorial Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | NLCO evaluates large language models for combinatorial optimization (CO) . existing evaluations emphasize relatively simple reasoning competencies . |
| Approach: | They propose a combinatorial optimization benchmark that evaluates large language models on CO reasoning. |
| Outcome: | The proposed model can handle combinatorial optimization without writing code or calling external solvers. |
SingaKids: A Multilingual Multimodal Dialogic Tutor for Language Learning (2025.acl-industry)
Copied to clipboard
Zhengyuan Liu, Geyu Lin, Hui Li Tan, Huayun Zhang, Yanfeng Lu, Xiaoxue Gao, Stella Xin Yin, Sun He, Hock Huan Goh, Lung Hsiang Wong, Nancy F. Chen
| Challenge: | Empirical studies show that SingaKids provides effective dialogic teaching, benefiting learners at different performance levels. |
| Approach: | They propose a dialogic tutor designed to facilitate language learning through picture description tasks. |
| Outcome: | Empirical studies show that SingaKids provides effective dialogic teaching, benefiting learners at different performance levels. |
Aligning Large Language Models with Implicit Preferences from User-Generated Content (2025.acl-long)
Copied to clipboard
Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang
| Challenge: | Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale. |
| Approach: | They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data. |
| Outcome: | The proposed framework transforms user-generated content into user queries and generates responses from the policy model. |
AdaTooler-V: Adaptive Tool-Use for Images and Videos (2026.findings-acl)
Copied to clipboard
Chaoyang Wang, Kaituo Feng, Dongyang Chen, Zhongyu Wang, Zhixun Li, Sicheng Gao, Meng Meng, Xu Zhou, Manyuan Zhang, Yuzhang Shang, Xiangyu Yue
| Challenge: | Existing models exhibit blind tool-use reasoning patterns, which significantly increases inference overhead and degrades model performance. |
| Approach: | They propose an MLLM that performs adaptive tool-use by determining whether a visual problem truly requires tools. |
| Outcome: | The proposed model outperforms existing methods in visual reasoning tasks. |
Towards the Law of Capacity Gap in Distilling Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Language model (LM) distillation aims at distilling knowledge in a large teacher LM to a small student one. |
| Approach: | They propose to use the law of capacity gap to distill knowledge from a large teacher to a small student model. |
| Outcome: | The proposed model outperforms other language models on a larger scale by using the law of capacity gap inducted from a preliminary study on small-scale (3B) LMs. |
Generate-then-Ground in Retrieval-Augmented Generation for Multi-hop Question Answering (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to solve multi-hop question are constrained by the retriever and the noise in the retrieved documents. |
| Approach: | They propose a framework that integrates parametric knowledge of large language models with external documents to solve a multi-hop question. |
| Outcome: | The proposed framework is based on the parametric knowledge of LLMs and external documents to solve a multi-hop question. |
DISCO: Distilling Counterfactuals with Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | high-quality counterfactual data is scarce for most tasks and not easily generated at scale. |
| Approach: | They propose a method for automatically generating high-quality counterfactual data at scale . they use a large general language model to generate phrasal perturbations and filter them . |
| Outcome: | The proposed method is task-agnostic and can be applied to the task of natural language inference. |
ServImage: An Image Generation and Editing Benchmark from Real-world Commercial Imaging Services (2026.acl-long)
Copied to clipboard
Fengxian Ji, Jingpu Yang, Zirui Song, Lang Gao, Junhong Liang, Zhenhao Chen, Jinghui Zhang, Xiuying Chen
| Challenge: | Recent image generation and editing models demonstrate robust adherence to instructions and high visual quality on academic benchmarks. |
| Approach: | They propose a benchmark that correlates image outputs with economic value in commercial design projects. |
| Outcome: | ServImage benchmarks show image generation models perform well on academic benchmarks but are uncertain on commercial projects. |
Iterative Document Representation Learning Towards Summarization with Polishing (D18-1)
Copied to clipboard
| Challenge: | Existing summarization methods read through document only once to generate a document representation, resulting in a sub-optimal representation. |
| Approach: | They propose an iterative model for supervised extractive text summarization which polishes the document representation on many passes through the document. |
| Outcome: | The proposed model outperforms state-of-the-art extractive systems on CNN/DailyMail and DUC2002 datasets. |
BlendFilter: Advancing Retrieval-Augmented Large Language Models via Query Generation Blending and Knowledge Filtering (2024.emnlp-main)
Copied to clipboard
Haoyu Wang, Ruirui Li, Haoming Jiang, Jinjin Tian, Zhengyang Wang, Chen Luo, Xianfeng Tang, Monica Cheng, Tuo Zhao, Jing Gao
| Challenge: | Retrieval-augmented Large Language Models struggle with complex inputs and noisy knowledge retrieval hindering model effectiveness. |
| Approach: | They propose a query generation method that integrates query generation blending with knowledge filtering to enhance retrieval-augmented LLMs. |
| Outcome: | The proposed approach surpasses state-of-the-art benchmarks on open-domain question answering benchmarks. |
SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utility (2026.acl-long)
Copied to clipboard
Xuyang Zhi, Peilun Zhou, Chengqiang Lu, Hang Lv, Yiwei Liang, Rongyang Zhang, Yan Gao, null Yiwu, Yao Hu, Hongchao Gu, Defu Lian, Hao Wang, Enhong Chen
| Challenge: | Large Language Models (LLMs) are shifting the focus from single verifiable tasks toward complex, open-ended real-world scenarios. |
| Approach: | They propose a framework that automatically adjusts reward weights and data importance to synchronize learning intent with data utility for optimal performance. |
| Outcome: | The proposed framework improves model capabilities across all domains and scales. |
DialoGPS: Dialogue Path Sampling in Continuous Semantic Space for Data Augmentation in Multi-Turn Conversations (2023.acl-long)
Copied to clipboard
| Challenge: | a context leads to various responses, and a response answers multiple contexts. |
| Approach: | They propose a method that augments open-domain dialogue generation from a many-to-many perspective. |
| Outcome: | The proposed method can augment open-domain dialogue generation tasks with automatic and human evaluation. |
Beyond Surface Simplicity: Revealing Hidden Reasoning Attributes for Precise Commonsense Diagnosis (2025.acl-long)
Copied to clipboard
| Challenge: | Existing commonsense question answering benchmarks often treat these aspects in isolation, resulting in evaluation accuracy differences of up to 24.8% across different difficulty levels. |
| Approach: | They propose a framework that reveals hidden reasoning attributes behind commonsense questions by leveraging the knowledge generated during the reasoning process. |
| Outcome: | The proposed framework reveals hidden reasoning attributes behind commonsense questions by leveraging the knowledge generated during the reasoning process. |
Unlocking LLMs’ Self-Improvement Capacity with Autonomous Learning for Domain Adaptation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models that use self-supervised and instruction fine-tuning can be trained using unlabeled corpora. |
| Approach: | They propose to use unlabeled target corpora to adapt large language models to new domains . they propose to employ self-supervised pre-training and instruction fine-tuning methods . |
| Outcome: | The proposed model can adapt to new domains using only a large amount of unlabeled target corpora. |
Self-Renewal Prompt Optimizing with Implicit Reasoning (2024.findings-emnlp)
Copied to clipboard
Zihan Liang, Ben Chen, Zhuoran Ran, Zihan Wang, Huangyu Dai, Yufei Ma, Dehong Gao, Xiaoyan Cai, Libin Yang
| Challenge: | Recent advances in NLP have been driven by the development of Large Language Models (LLMs). |
| Approach: | They propose a self-renewal approach to optimize LLM outputs to better align with human preferences without supervised fine-tuning. |
| Outcome: | The proposed approach improves outputs to better align with human preferences across LLMs and tasks without supervised fine-tuning. |
Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach (2021.emnlp-main)
Copied to clipboard
Simiao Zuo, Chen Liang, Haoming Jiang, Xiaodong Liu, Pengcheng He, Jianfeng Gao, Weizhu Chen, Tuo Zhao
| Challenge: | Existing approaches to adversarial regularization treat adversarials and defending players equally, which is undesirable because only the defending player contributes to the generalization performance. |
| Approach: | They propose a method which formulates adversarial regularization as a Stackelberg game and induces a competition between a leader and a follower. |
| Outcome: | The proposed method outperforms existing adversarial regularization baselines on a set of machine translation and natural language understanding tasks. |
ALERT: An LLM-powered Benchmark for Automatic Evaluation of Recommendation Explanations (2025.naacl-long)
Copied to clipboard
Yichuan Li, Xinyang Zhang, Chenwei Zhang, Mao Li, Tianyi Liu, Pei Chen, Yifan Gao, Kyumin Lee, Kaize Ding, Zhengyang Wang, Zhihan Zhang, Jingbo Shang, Xian Li, Trishul Chilimbi
| Challenge: | Existing benchmarks for recommendation explanation evaluation lack item diversity and user preferences data. |
| Approach: | They propose a model-agnostic recommendation explanation evaluation benchmark based on Amazon e-commerce categories with implicit preferences . they propose two novel automatic evaluators that enable scalable and human-preference aligned evaluation of explanations . |
| Outcome: | The proposed model-agnostic evaluation benchmark outperforms existing methods in a variety of domains. |
Dual-Phase Accelerated Prompt Optimization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for prompt optimization make light of the importance of high-quality initialization and the identification of effective directions. |
| Approach: | They propose a method which uses a meta-instruction to generate high-quality initial prompts and iteratively optimize them at the sentence level. |
| Outcome: | The proposed method achieves consistent accuracy gain over baselines with less than five optimization steps. |
Title2Event: Benchmarking Open Event Extraction with a Large-scale Chinese Title Dataset (2022.emnlp-main)
Copied to clipboard
Haolin Deng, Yanan Zhang, Yangfan Zhang, Wangyang Ying, Changlong Yu, Jun Gao, Wei Wang, Xiaoling Bai, Nan Yang, Jin Ma, Xiang Chen, Tianhua Zhou
| Challenge: | Existing EE datasets define fixed event types and design specific schemas for each of them, failing to cover diverse events emerging from the online text. |
| Approach: | They propose to use a sentence-level dataset to benchmark Open Event Extraction without restricting event types. |
| Outcome: | The proposed dataset contains more than 42,000 news titles in 34 topics collected from Chinese web pages. |
The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural Language Understanding (2020.acl-demos)
Copied to clipboard
Xiaodong Liu, Yu Wang, Jianshu Ji, Hao Cheng, Xueyun Zhu, Emmanuel Awa, Pengcheng He, Weizhu Chen, Hoifung Poon, Guihong Cao, Jianfeng Gao
| Challenge: | MT-DNN is an open-source natural language understanding toolkit . it allows researchers and developers to train customized deep learning models . |
| Approach: | They present MT-DNN, an open-source natural language understanding toolkit . it is designed to facilitate rapid customization for a broad spectrum of NLU tasks . MT supports multi-task knowledge distillation, which can substantially compress a deep neural model without significant performance drop. |
| Outcome: | The proposed model can significantly compress a large model without significant performance drop. |
Can We Predict Before Executing Machine Learning Agents? (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to scientific discovery rely on expensive physical execution . a Generate-Execute-Feedback paradigm is costly and slow . |
| Approach: | They propose to internalize execution priors to substitute costly runtime checks with instantaneous predictive reasoning, drawing inspiration from World Models. |
| Outcome: | The proposed framework achieves 61.5% accuracy and robust confidence calibration when primed with a Verified Data Analysis Report. |
RARR: Researching and Revising What Language Models Say, Using Language Models (2023.acl-long)
Copied to clipboard
Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, Kelvin Guu
| Challenge: | Language models (LMs) excel at many tasks but often produce unsupported or misleading content. |
| Approach: | They propose a system that finds attribution for any text generation model and post-edits it to fix unsupported content. |
| Outcome: | The proposed system improves attribution while preserving the original output. |
Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (2025.naacl-long)
Copied to clipboard
Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang
| Challenge: | Existing LLMs often rely on complex prompting or extensive fine-tuning to introduce new capabilities while preserving strong generalizability. |
| Approach: | They propose a large-scale pre-training corpus to enhance LLM agents' capabilities . they use 103B agent-specific data encompassing 76,537 APIs . |
| Outcome: | The proposed training corpus outperforms open-source LLMs and commercial LLM agents on three agent benchmarks. |
LLMs Trust Humans More, That’s a Problem! Unveiling and Mitigating the Authority Bias in Retrieval-Augmented Generation (2025.acl-long)
Copied to clipboard
Yuxuan Li, Xinwei Guo, Jiashi Gao, Guanhua Chen, Xiangyu Zhao, Jiaxin Zhang, Quanying Liu, Haiyan Wu, Xin Yao, Xuetao Wei
| Challenge: | Large language models (LLMs) generate outputs that stray from user input or contravene established knowledge. |
| Approach: | They propose a new phenomenon, Authority Bias, where LLMs favor one knowledge source over the other . they propose atomic information that generates conflicts and a Conflict Detection Enhanced Query framework . |
| Outcome: | The proposed framework reduces Authority bias in large language models . it detects conflicts, performs credibility assessment on conflicting paragraphs, and detects perturbed text . |
Distributed LLM Serving on Consumer-Grade GPUs by Reconciling Computation and Communication (2025.findings-emnlp)
Copied to clipboard
Lewei Jin, Kui Zhang, Yongqi Chen, null Zhuoyifan, Renjie Li, Yi Gao, Bowei Yang, Zhengong Cai, Wei Dong
| Challenge: | Large language models are reshaping internet services, and serving them is costly. |
| Approach: | They propose an efficient distributed LLM serving system that splits prefill and decode requests into smaller chunks . |
| Outcome: | The proposed system reduces TTFT, TPOT, and latency compared to the state-of-the-art system. |
CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Space (2025.emnlp-main)
Copied to clipboard
Yong Zhao, Kai Xu, Zhengqiu Zhu, Yue Hu, Zhiheng Zheng, Yingfeng Chen, Yatai Ji, Chen Gao, Yong Li, Jincai Huang
| Challenge: | Embodied Question Answering (EQA) tasks are primarily focused on indoor environments, leaving the complexities of urban settings unexplored. |
| Approach: | They propose a task where an embodied agent answers open-vocabulary questions in dynamic city spaces. |
| Outcome: | The proposed agent achieves 60.7% of human-level answering accuracy compared to baselines . the proposed agent outperforms existing agents in open-ended city spaces . |
Textual Backdoor Attacks Can Be More Harmful via Two Simple Tricks (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing textual backdoor attacks are vulnerable to backdoors . researchers add extra training task to distinguish poisoned and clean data . |
| Approach: | They propose two tricks that make existing backdoor attacks much more harmful . first trick is to add an extra task to distinguish poisoned and clean data . second trick is using all the clean training data rather than the original clean data. |
| Outcome: | The proposed tricks can significantly improve attack performance in three tough situations including clean data fine-tuning, low-poisoning-rate, and label-consistent attacks. |
Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT - A Text-to-SQL Parsing Comparison (2023.findings-emnlp)
Copied to clipboard
| Challenge: | a number of open-source large language models claim to be performing better than commercial ones . however, these models fall short of the performance achieved by closed-source models like GPT-3.5 . |
| Approach: | They evaluate six popular large language models against each other to evaluate their performance . authors say open-source models are not as effective as those built by commercial models . |
| Outcome: | a new set of models claim to match or surpass the language understanding abilities of commercial models . the results show that the models performed far below the performance of closed-source models compared to open-source ones . |
Rethinking Multi-Modal Alignment in Multi-Choice VideoQA from Feature and Sample Perspectives (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to VideoQA focus on utilizing frame- or object-level visual representations, but they neglect visual-language interactions. |
| Approach: | They propose to break down video into trajectories and first leverage trajectory feature in VideoQA to enhance alignment between two modalities. |
| Outcome: | The proposed method outperforms all the state-of-the-art models on the NExT-QA benchmark. |
Should You Mask 15% in Masked Language Modeling? (2023.eacl-main)
Copied to clipboard
| Challenge: | Masked language models (MLMs) traditionally mask 15% of tokens due to the belief that more masking would leave insufficient context to learn good representations. |
| Approach: | They revisit the 15% masking rate of MLMs to examine the role of masking in linguistic training. |
| Outcome: | The proposed masking rate outperforms BERT-large size models on GLUE and SQUAD while maintaining 95% accuracy. |
Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP (2022.emnlp-main)
Copied to clipboard
| Challenge: | Textual adversarial samples are often misrepresented in research on security, evaluation, explainability, and data augmentation. |
| Approach: | They propose to use adversarial samples to evaluate their methods on security tasks to demonstrate the real-world concerns rather than developing impractical methods. |
| Outcome: | The proposed method has higher practical value than the current benchmark. |
Are LLM-based Evaluators Confusing NLG Quality Criteria? (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that LLMs confuse evaluation criteria, which reduces their reliability. |
| Approach: | They propose a hierarchical classification system for 11 common aspects with corresponding different evaluation criteria. |
| Outcome: | The proposed system is based on 11 common aspects with different evaluation criteria. |
The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation (2022.tacl-1)
Copied to clipboard
Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, Angela Fan
| Challenge: | a lack of good evaluation benchmarks hinders progress in low-resource and multilingual machine translation . despite advances in translation quality for a handful of languages, many low-source languages are not even supported by most popular translation engines. |
| Approach: | They propose a high-quality evaluation benchmark for machine translation using 3001 sentences from Wikipedia . they aim to improve evaluation of models on long tail of low-resource languages . |
| Outcome: | The proposed evaluation benchmarks are based on 3001 sentences extracted from Wikipedia . the results show that the models can be used to evaluate multilingual systems . |
Exploring the Universal Vulnerability of Prompt-based Learning Paradigm (2022.findings-naacl)
Copied to clipboard
| Challenge: | Prompt-based learning inherits the vulnerability from pre-training, where model predictions can be misled by inserting triggers into the text. |
| Approach: | They propose a potential solution to mitigate this vulnerability by injecting triggers into pre-trained language models using only plain text. |
| Outcome: | The proposed learning paradigm inherits the vulnerability from the pre-training stage . it can totally control or severely decrease the performance of prompt-based models . |
Stability Implies Redundancy: Delta Attention Selective Halting for Efficient Long-Context Prefilling (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods to reduce sequence length rely on heuristics that break compatibility with hardware-efficient kernels like FlashAttention. |
| Approach: | They propose a method that selectively halts stabilized tokens by monitoring layer-wise update dynamics of the self-attention mechanism. |
| Outcome: | The proposed method can reduce prefill complexity while preserving model accuracy and hardware efficiency. |
DataSeer: A Manager-Centric Collaborative Multi-Agent Framework with Multi-Branch Reasoning for Automated Insight Discovery (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for automated insight discovery lack contextual coherence and coverage due to single-path exploration. |
| Approach: | They propose a Manager-Centric Collaborative Framework that integrates planner and executor . it ensures cross-episode contextual coherence and allows for adaptive sub-goal generation . |
| Outcome: | The proposed framework outperforms baselines on InsightBench and Inseval. |
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
MMEvol: Empowering Multimodal Large Language Models with Evol-Instruct (2025.findings-acl)
Copied to clipboard
Run Luo, Haonan Zhang, Longze Chen, Ting-En Lin, Xiong Liu, Yuchuan Wu, Min Yang, Yongbin Li, Minzheng Wang, Pengpeng Zeng, Lianli Gao, Heng Tao Shen, Yunshui Li, Hamid Alinejad-Rokny, Xiaobo Xia, Jingkuan Song, Fei Huang
| Challenge: | a new framework for image-text instruction data evolution improves MLLM performance . lack of high-quality instruction data remains a major bottleneck in ML modeling . |
| Approach: | They propose a multimodal instruction data evolution framework that iteratively enhances data quality through fine-grained perception, cognitive reasoning, and interaction evolution. |
| Outcome: | The proposed approach improves MLLM performance in nine vision-language tasks while using significantly less data. |
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data (2024.findings-naacl)
Copied to clipboard
Ziyi Yang, Mahmoud Khademi, Yichong Xu, Reid Pryzant, Yuwei Fang, Chenguang Zhu, Dongdong Chen, Yao Qian, Xuemei Gao, Yi-Ling Chen, Robert Gmyr, Naoyuki Kanda, Noel Codella, Bin Xiao, Yu Shi, Lu Yuan, Takuya Yoshioka, Michael Zeng, Xuedong Huang
| Challenge: | i-Code V2 is one of the first models capable of generating natural language from any combination of Vision, Language, and Speech data. |
| Approach: | They propose to create a model that can generate natural language from any combination of Vision, Language, and Speech data. |
| Outcome: | i-Code V2 matches or outperforms state-of-the-art single- and dual-modality baselines on 7 multimodal tasks. |
Selection and Generation: Learning towards Multi-Product Advertisement Post Generation (2020.emnlp-main)
Copied to clipboard
| Challenge: | E-commerce websites have billions of products, so it is impossible to write all copywriting manually. |
| Approach: | They propose a model to generate an AD post using a select network and a MGenNet network to generate a post including selected products. |
| Outcome: | The proposed model achieves impressive performance on a large-scale real-world AD post dataset. |
Confidence-Aware Ranker Ensembles for Robust In-Context Knowledge Editing (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models excel at factual recall, but can propagate stale or incorrect knowledge. |
| Approach: | They propose a feature-weighted ensemble for in-context knowledge editing that calibrates three heterogeneous rankers and extracts simple confidence features from each ranker. |
| Outcome: | The proposed method achieves 88.33% Edit-Success Rate over the best single retriever . it improves edit accuracy without touching model weights and approaches oracle upper bound (91%). |
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model (2023.findings-acl)
Copied to clipboard
Xiao Wang, Weikang Zhou, Qi Zhang, Jie Zhou, SongYang Gao, Junzhe Wang, Menghan Zhang, Xiang Gao, Yun Wen Chen, Tao Gui
| Challenge: | Pretrained language models have achieved remarkable success in various natural language processing tasks. |
| Approach: | They propose to use end-task knowledge to select a tiny subset of pretraining corpus to influence performance. |
| Outcome: | The proposed model outperforms pretrained models on eight datasets covering four domains with 0.45% of the data and a three-orders-of-magnitude lower computational cost. |
KPatch: Knowledge Patch to Pre-trained Language Model for Zero-Shot Stance Detection on Social Media (2024.lrec-main)
Copied to clipboard
Shuohao Lin, Wei Chen, Yunpeng Gao, Zhishu Jiang, Mengqi Liao, Zhiyu Zhang, Shuyuan Zhao, Huaiyu Wan
| Challenge: | Existing knowledge injection methods fail to understand the semantics of tweets . |
| Approach: | They propose a method to flexibly inject knowledge into a pre-trained language model and adaptively expand tweets context. |
| Outcome: | The proposed method is based on two training stages to flexibly inject knowledge into the pre-trained language model and adaptively expand tweets context. |
Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale (2024.emnlp-main)
Copied to clipboard
Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Chen, Xidong Wang, Zhenyang Cai, Ke Ji, Xiang Wan, Benyou Wang
| Challenge: | Multimodal large language models (MLLMs) lack visual knowledge in medical applications due to data privacy concerns and high annotation costs. |
| Approach: | They refined medical image-text pairs from PubMed and employed MLLMs (GPT-4V) to denoise and reformat the data. |
| Outcome: | The proposed model significantly improves the MMMU Health & Medicine track and shows that it can be used in multimodal scenarios. |
Beyond Black-Box Interventions: Latent Probing for Faithful Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
Linfeng Gao, Qinggang Zhang, Baolong Bi, Bo Zeng, Zheng Yuan, Zerui Chen, Zhimin Wei, Shenghua Liu, Linlong Xu, Longyue Wang, Weihua Luo, Jinsong Su
| Challenge: | Existing approaches to improve contextual faithfulness treat the LLM as a black box, generating responses that are inconsistent with the provided context. |
| Approach: | They propose a framework for faithful RAG that operates in three stages: (i) fine-grained knowledge pruning to filter irrelevant context, (ii) latent conflict probing to identify hard conflicts in the model’s latent space, and (iv) conflict-aware attention to modulate attention heads toward faithful context integration. |
| Outcome: | Experiments show that ProbeRAG significantly improves both accuracy and contextual faithfulness. |
Retrievals Can Be Detrimental: Unveiling the Backdoor Vulnerability of Retrieval-Augmented Diffusion Models (2026.acl-long)
Copied to clipboard
| Challenge: | Retrieval-augmented diffusion models (RDMs) have been developed to enhance performance with reduced parameters. |
| Approach: | They propose to integrate retrieval-augmented diffusion models with Retrieval-augmented generation (RAG) that enhances performance with reduced parameters. |
| Outcome: | The proposed framework achieves outstanding attack effects while maintaining benign utility. |
Scientific Paper Extractive Summarization Enhanced by Citation Graphs (2022.emnlp-main)
Copied to clipboard
| Challenge: | citation graphs can be used to extract scientific papers under different conditions. |
| Approach: | They propose a multi-granularity unsupervised summarization model that fine tunes a pre-trained encoder model on the citation graph by link prediction tasks. |
| Outcome: | The proposed model outperforms baseline models on a public benchmark dataset. |
Few-Shot Named Entity Recognition: An Empirical Baseline Study (2021.emnlp-main)
Copied to clipboard
Jiaxin Huang, Chunyuan Li, Krishan Subudhi, Damien Jose, Shobana Balakrishnan, Weizhu Chen, Baolin Peng, Jianfeng Gao, Jiawei Han
| Challenge: | Existing methods to build named entity recognition systems with limited labeled data are lacking. |
| Approach: | They propose three orthogonal schemes to build named entity recognition systems when labeled data is limited. |
| Outcome: | The proposed NER systems outperform existing methods on few-shot and training-free settings. |
MES-RAG: Bringing Multi-modal, Entity-Storage, and Secure Enhancements to RAG (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to retrieve entity information are limited by document level retrieval and intermingled storage of information from different entities. |
| Approach: | They propose a framework that enhances entity-specific query handling . MES-RAG introduces proactive security measures that ensure system integrity . |
| Outcome: | Experimental results show that MES-RAG improves accuracy and recall . the framework can be integrated into existing RAG architectures . |
Learning to Use Tools via Cooperative and Interactive Agents (2024.findings-emnlp)
Copied to clipboard
Zhengliang Shi, Shen Gao, Xiuyi Chen, Yue Feng, Lingyong Yan, Haibo Shi, Dawei Yin, Pengjie Ren, Suzan Verberne, Zhaochun Ren
| Challenge: | Existing methods for large language models (LLMs) use one agent to iterate and execute tools, but they suffer from performance degradation when addressing practical tasks. |
| Approach: | They propose a tool learning framework that coordinates three specialized agents for tool selection, tool execution, and action calibration separately. |
| Outcome: | The proposed framework outperforms baseline models on three datasets with 14% higher success rate. |
“What Do You Mean by That?” A Parser-Independent Interactive Approach for Enhancing Text-to-SQL (2020.emnlp-main)
Copied to clipboard
| Challenge: | In Natural Language Interfaces to Databases systems, text-to-SQL parsers allow users to query databases by using natural language questions. |
| Approach: | They propose a parser-independent interactive approach that interacts with users using multi-choice questions and can easily work with arbitrary parsers. |
| Outcome: | The proposed approach improves performance with limited interaction turns by using simulation and human evaluation on two cross-domain datasets with five state-of-the-art parsers. |
DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models (2024.acl-long)
Copied to clipboard
Damai Dai, Chengqi Deng, Chenggang Zhao, R.x. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.k. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, Wenfeng Liang
| Challenge: | Mixture-of-Experts (MoE) architectures face challenges in ensuring expert specialization . despite the promising performance, scaling language models to an extremely large scale is associated with exceedingly high computational costs. |
| Approach: | They propose an architecture that allows for ultimate expert specialization by segmenting experts into mN ones and activating mK from them. |
| Outcome: | The proposed architecture achieves comparable performance with GShard with 2B parameters and computation. |
Reasoning Like Program Executors (2022.emnlp-main)
Copied to clipboard
Xinyu Pi, Qian Liu, Bei Chen, Morteza Ziyadi, Zeqi Lin, Qiang Fu, Yan Gao, Jian-Guang Lou, Weizhu Chen
| Challenge: | Existing language models are inadequate in reasoning, according to studies . a new reasoning pre-training paradigm is based on pretraining language models with programs . |
| Approach: | They propose a reasoning pre-training paradigm that empowers language models to harvest reasoning knowledge possessed by program executors. |
| Outcome: | The proposed reasoning pre-training paradigm can boost models' reasoning skills . it can be instantiated by different kinds of program executors and run on a single database . |
Token-wise Curriculum Learning for Neural Machine Translation (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches to Neural Machine Translation (NMT) require sampling sufficient amounts of “easy” samples from training data at the early stage of training. |
| Approach: | They propose a token-wise curriculum learning approach that creates sufficient amounts of easy samples from training data. |
| Outcome: | The proposed approach outperforms baselines on five language pairs on low-resource languages. |
Empowering Large Language Model for Continual Video Question Answering with Collaborative Prompting (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing VideoQA models struggle to adapt to new questions or tasks posed by newly available content. |
| Approach: | They propose a continual learning framework that fine-tunes a large language model for a sequence of tasks and integrates specific question constraint prompting, knowledge acquisition prompting and visual temporal awareness prompting. |
| Outcome: | The proposed model achieves 55.14% accuracy on both NExT-QA and DramaQA datasets and 71.24% accuracy for DramaQA. |
Under the Shadow of Babel: How Language Shapes Reasoning in LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | linguistic relativity suggests that the structure of language shapes cognitive patterns . large language models internalize the habitual logical structures embedded in different languages, authors say . |
| Approach: | a study introduces a bilingual dataset for causal reasoning in Chinese and English. |
| Outcome: | a new study shows that large language models internalize reasoning biases shaped by language . the model internalizes language-specific preferences and rigidly applies them to atypical inputs, the study shows . |
UMSE: Unified Multi-scenario Summarization Evaluation (2023.findings-acl)
Copied to clipboard
| Challenge: | Summarization quality evaluation is a non-trivial task in text summarization. |
| Approach: | They propose a unified multi-scenario summarization evaluation model that shares cross-sceenario knowledge and uses a self-supervised training paradigm to optimize the model without extra human labeling. |
| Outcome: | The proposed model can achieve comparable performance with existing methods for three evaluation scenarios. |
Document Segmentation Matters for Retrieval-Augmented Generation (2025.findings-acl)
Copied to clipboard
Zhitong Wang, Cheng Gao, Chaojun Xiao, Yufei Huang, Shuzheng Si, Kangyang Luo, Yuzhuo Bai, Wenhao Li, Tangjian Duan, Chuancheng Lv, Guoshan Lu, Gang Chen, Fanchao Qi, Maosong Sun
| Challenge: | Existing rule-based chunking methods lead to suboptimal splits, where overly large chunks introduce irrelevant information and small chunks lack semantic coherence. |
| Approach: | They propose a method that leverages document summaries as pseudo-instructions to guide chunking by computing semantic similarity between sentences and the summary. |
| Outcome: | Experiments on multiple open-domain question-answering benchmarks show that PIC significantly improves retrieval accuracy (Hits@k) and end-to-end QA performance (Exact Match) without any additional training. |
UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations (2025.acl-long)
Copied to clipboard
Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, Bing Yin, Meng Jiang
| Challenge: | Existing conversational search systems are usually built with two different models . this separation restricts the system from leveraging the model's intrinsic knowledge simultaneously . Existing studies for developing unified models cannot fully address the aspects of understanding conversational context, managing retrieval independently, and generating responses. |
| Approach: | They propose to unify dense retrieval and response generation for large language models in conversation by fine-tuning and mitigating data discrepancy. |
| Outcome: | The proposed model can outperform existing models on five conversational search datasets and reduce inconsistency risks while mitigating data discrepancy. |
AgentGym: Evaluating and Training Large Language Model-based Agents across Diverse Environments (2025.acl-long)
Copied to clipboard
Zhiheng Xi, Yiwen Ding, Wenxiang Chen, Boyang Hong, Honglin Guo, Junzhe Wang, Xin Guo, Dingwen Yang, Chenyang Liao, Wei He, Songyang Gao, Lu Chen, Rui Zheng, Yicheng Zou, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, Yu-Gang Jiang
| Challenge: | Large language models (LLMs) are promising foundations to build generally-capable agents . however, the community lacks a unified interactive framework that covers diverse environments for comprehensive evaluation of agents. |
| Approach: | They propose a framework that features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
| Outcome: | The proposed framework features 7 real-world scenarios, 14 environments, and 89 tasks for unified, real-time, and concurrent agent interaction. |
Coarse-to-fine Few-shot Learning for Named Entity Recognition (2023.findings-acl)
Copied to clipboard
Ruotian Ma, Zhang Lin, Xuanting Chen, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen
| Challenge: | Existing few-shot NER solutions do not consider sub-class discrimination and various granularity of new classes during coarse training. |
| Approach: | They propose a method that uses a cluster-based prototype loss to learn group-wise discriminative representations of coarse-grained classes and a mixture prototype loss for learning the representations. |
| Outcome: | The proposed method shows superior performance over baseline methods on in-domain and cross-domain settings with various target granularity. |
DialMed: A Dataset for Dialogue-based Medication Recommendation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies on medication recommendation mainly rely on EHRs, but some details of interactions between doctors and patients may be ignored or omitted in EHR. |
| Approach: | They propose to use medical dialogues to recommend medications with medical dialogue data . they propose to model dialogue structure and disease knowledge aware network . |
| Outcome: | The proposed method is a promising solution to recommend medications with medical dialogues. |
Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work on calibration and uncertainty quantification focuses on individual models, overlooking the potential of model diversity. |
| Approach: | They propose a method that uses Jensen-Shannon Divergence to identify and aggregate well-calibrated subsets of large language models (LLMs) to improve calibration. |
| Outcome: | The proposed method improves accuracy on binary prediction tasks compared to single-model and naive ensemble baselines. |
Rectified Sparse Attention for Efficient Long-Sequence Generation (2026.findings-acl)
Copied to clipboard
Yutao Sun, Tianzhu Ye, Li Dong, Yuqing Xia, Jian Chen, Yizhao Gao, Shijie Cao, Jianyong Wang, Furu Wei
| Challenge: | Recent sparse decoding methods improve efficiency but suffer from KV cache misalignment, resulting in performance degradation. |
| Approach: | They propose a method that combines block-sparse attention with periodic dense rectification to bound error accumulation and preserve alignment with the pretraining distribution. |
| Outcome: | Experiments on math reasoning, language modeling, and retrieval tasks show that ReSA achieves near-lossless generation quality with significantly improved efficiency. |
Label-Aware Double Transfer Learning for Cross-Specialty Medical Named Entity Recognition (N18-1)
Copied to clipboard
Zhenghui Wang, Yanru Qu, Liheng Chen, Jian Shen, Weinan Zhang, Shaodian Zhang, Yimei Gao, Gen Gu, Ken Chen, Yong Yu
| Challenge: | NER is a fundamental problem for medical text mining because of the difference of specialties and cost of human annotation. |
| Approach: | They propose a label-aware double transfer learning framework for medical NER from electronic medical records. |
| Outcome: | The proposed framework improves accuracy over strong baselines on 12 cross-specialty NER tasks. |
Audio Jailbreak: An Open Comprehensive Benchmark for Jailbreaking Large Audio-Language Models (2026.acl-long)
Copied to clipboard
Zirui Song, Qian Jiang, Mingxuan Cui, Mingzhe Li, Lang Gao, Zeyu Zhang, Zixiang Xu, Yanbo Wang, Guangxian Ouyang, Zhenhao Chen, Xiuying Chen
| Challenge: | a recent study evaluated large audio-language models against jailbreak attacks . a new benchmark is being developed to evaluate LAM safety against jailbreaking attacks based on temporal and semantic nature of speech . |
| Approach: | They propose a benchmark to evaluate LAM jailbreak vulnerabilities in adversarial audio prompts . they use a dataset of 1,495 adversarials to evaluate their performance . |
| Outcome: | The proposed benchmark evaluates state-of-the-art LAMs against jailbreak attacks . it demonstrates that even small, semantically preserved perturbations can reduce safety . |
MMAC: A Multilingual, Multimodal Alignment Framework for Cultural Grounding Evaluation (2026.acl-long)
Copied to clipboard
Weihua Zheng, Zhengyuan Liu, Tanmoy Chakraborty, Weiwen Xu, Xiaoxue Gao, Bryan Chen Zhengyu Tan, Bowei Zou, Chang Liu, Yujia Hu, Xing Xie, Xiaoyuan Yi, Jing Yao, Chaojun Wang, Long Li, Rui Liu, Huiyao Liu, Koji Inoue, Ryuichi Sumida, Tatsuya Kawahara, Fan Xu, Lingyu Ye, Wei Tian, Dongjun Kim, Jimin Jung, Jaehyung Seo, Nadya Yuki Wangsajaya, Pham Minh Duc, Ojasva Saxena, Palash Nandi, Xiyan Tao, Wiwik Karlina, Tuan Luong, Keertana Arun Vasan, Roy Ka-Wei Lee, Nancy F. Chen
| Challenge: | Existing models lack cultural alignment across modalities and languages . a new framework to assess cultural awareness across linguistics and languages is needed . |
| Approach: | They propose a framework that integrates tri-modally aligned cultural benchmarks and a five-dimensional evaluation protocol to assess cross-country awareness disparities. |
| Outcome: | The proposed framework assesses cultural awareness disparities across modalities and languages . it is the first dataset aligned at the input level across text, image, and speech . |
BioGen: Generating Biography Summary under Table Guidance on Wikipedia (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for summarizing text have not captured the salient information from an article. |
| Approach: | They propose a table-guided abstractive biography summarization that utilizes factual tables to capture important information and generate a summary of a biography. |
| Outcome: | The proposed method is the first large-scale biography summarization dataset with tables. |
TaxPraBen: A Scalable Benchmark for Structured Evaluation of LLMs in Chinese Real-World Tax Practice (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models excel in general domains but lack real-world practical capabilities. |
| Approach: | They propose a benchmark for Chinese taxation practice that combines 10 traditional application tasks with 3 pioneering real-world scenarios. |
| Outcome: | The proposed benchmark combines 10 traditional tasks with 3 pioneering real-world scenarios. |
NICE: Neural Image Commenting with Empathy (2021.findings-emnlp)
Copied to clipboard
Kezhen Chen, Qiuyuan Huang, Daniel McDuff, Xiang Gao, Hamid Palangi, Jianfeng Wang, Kenneth Forbus, Jianfeng Gao
| Challenge: | Emotion and empathy are examples of human qualities lacking in many human-machine interactions. |
| Approach: | They propose to generate images with human-generated comments with enhanced emotion and empathy while minimizing inappropriate or offensive outputs. |
| Outcome: | The proposed model generates more human-like and engaging image comments on two images with human-generated comments and human annotations while minimizing inappropriate or offensive outputs. |
emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing models for speech emotion recognition are not suitable for emotional tasks. |
| Approach: | They propose a universal speech emotion representation model that is pre-trained on open-source emotion data. |
| Outcome: | euphoria2vec outperforms state-of-the-art models and emotion specialist models . it shows consistent improvements among 10 different languages of speech emotion recognition datasets . |
CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward (2025.emnlp-main)
Copied to clipboard
Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chengqi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F. Wong, Songyang Zhang, Kai Chen
| Challenge: | Existing approaches lack robustness to handle complex edge cases and generalizability across different domains. |
| Approach: | They develop an accurate and lightweight verifier model for evaluation and outcome reward that matches unstructured outputs against standard answers. |
| Outcome: | The proposed model can process multiple answer types including multi-subproblems, formulas, and sequence answers while identifying abnormal/invalid responses. |
PACE: Prefix-Protected and Difficulty-Aware Compression for Efficient Reasoning (2026.findings-acl)
Copied to clipboard
Ruixiang Feng, Yuntao Wen, Silin Zhou, Ke Shi, Yifan Wang, Ran Le, Zhenwei An, Zongchao Chen, Chen Yang, Guangyue Peng, Yiming Jia, Dongsheng Wang, Tao Zhang, Lisi Chen, Yang Song, Shen Gao, Shuo Shang
| Challenge: | Existing LRMs often suffer from "overthinking" and excessively long reasoning traces . a dual-level framework for length compression of LRM is proposed . |
| Approach: | They propose a framework for prefix-protected and difficulty-aware compression under hierarchical supervision. |
| Outcome: | The proposed framework reduces token usage while improving accuracy on math benchmarks. |
Leveraging Unpaired Feedback for Long-Term LLM-based Recommendation Tuning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study highlights unpaired feedback as a key challenge for long-term LLM-based recommenders . unpaired user feedback is crucial for improving LLMs in dynamic user environments, authors say . |
| Approach: | They propose a framework that incorporates unpaired feedback into LLMs to improve long-term recommendation performance. |
| Outcome: | The proposed framework improves long-term recommendation performance by incorporating unpaired feedback without requiring paired supervision. |
SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel in code generation benchmarks, but these benchmarks focus on single-file scenarios with constrained context scope. |
| Approach: | They propose an open-source framework to effectively resolve GitHub issues using a code file retrieval module and a model-based code editing module. |
| Outcome: | The proposed approach achieves state-of-the-art performance on two GitHub benchmarks. |
Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization (2026.findings-acl)
Copied to clipboard
Mingkuan Zhao, Wentao Hu, Tianchen Huang, Yuheng Min, Suquan Chen, Yide Gao, Yanbo Zhai, Shuangyong Song, Xuelong Li
| Challenge: | Hallucinations in Large Language Models persist in critical domains where generated content diverges from contextual facts or logical constraints. |
| Approach: | They propose to generate hallucinations as orthogonal noise relative to the semantic manifold of the residual stream. |
| Outcome: | The proposed method achieves superior contextual faithfulness compared to state-of-the-art methods. |
Wasserstein Distance Regularized Sequence Representation for Text Matching in Asymmetrical Domains (2020.emnlp-main)
Copied to clipboard
| Challenge: | Asymmetrical text matching is a fundamental problem in information retrieval and natural language processing. |
| Approach: | They propose a method that regularizes features vectors projected from different domains . WD-Match can be used to improve different text matching methods . |
| Outcome: | The proposed method outperforms existing methods and benchmarks on four datasets. |
Anti-Length Shift: Dynamic Outlier Truncation for Training Efficient Reasoning Models (2026.acl-long)
Copied to clipboard
Wei Wu, Liyi Chen, Congxi Xiao, Tianfu Wang, Qimeng Wang, Chengqiang Lu, Yan Gao, null Yiwu, Yao Hu, Hui Xiong
| Challenge: | Existing efficient reasoning methods rely on explicit length penalties for excessive verbosity on simple queries. |
| Approach: | They propose a training-time intervention that selectively suppresses redundant tokens . they find length shift occurs when models generate unnecessary reasoning on trivial inputs - a phenomenon that is often unexplored . |
| Outcome: | The proposed method reduces inference token usage by 78% while increasing accuracy compared to the initial policy and surpasses state-of-the-art efficient reasoning methods. |
Towards Interpretable Tabular Reasoning: Enhancing LLM Reasoning on Tabular Data with Pre-Constructed Logic Graph (2026.acl-long)
Copied to clipboard
Lirong Gao, Zewei Yu, Zhongrui Yin, Qi Zhang, Yuke Zhu, Bo Zheng, Haobo Wang, Junbo Zhao, Gang Chen, Sheng Guo
| Challenge: | Tabular data is used in fields such as finance and healthcare due to its heterogeneity and complexity. |
| Approach: | They propose a Logic-Graph-Enhanced LLM Reasoning framework that integrates the strengths of tree-based models and LLMs to improve their interpretability. |
| Outcome: | The proposed framework outperforms tree-based models and state-of-the-art LLMs on tabular prediction tasks, achieving superior accuracy and interpretability. |
Summarizing Procedural Text: Data and Approach (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Procedural text summarization task is a popular task in the NLP field because of its long length and complexity. |
| Approach: | They propose a procedural text summarization task with two granularity . they propose an Entity-State Graph-based Summarizer (ESGS) which aggregates contextual information for each procedure. |
| Outcome: | The proposed model can summarize the entire procedural text or give an overview for each step or both . Experiments on two datasets confirm the proposed model's effectiveness. |
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation (2020.acl-demos)
Copied to clipboard
Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, Bill Dolan
| Challenge: | DIALOGPT is a large, tunable neural conversational response generation model . trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Approach: | They present a large, tunable neural conversational response generation model, DIALOGPT . the model is trained on 147M conversation-like exchanges extracted from Reddit comment chains . |
| Outcome: | The proposed model can generate more relevant, contentful and context-consistent responses than baseline systems. |
D2PCM:A Multi-Turn Dialogue Dataset with Personalized Contextual Memory (2026.findings-acl)
Copied to clipboard
| Challenge: | Conventional interactive algorithms have predominantly treated memory as a contextual element, neglecting the nuanced cognitive processes involved in individualized memory encoding and retrieval. |
| Approach: | They propose a multi-turn dialogue dataset with Personalized Contextual Memory to facilitate advanced research on personalized memory processing. |
| Outcome: | The proposed datasets provide a comprehensive benchmark to facilitate advanced research on personalized memory processing. |
ARM: An Alignment-and-Replacement Module for Chinese Spelling Check Based on LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Chinese Spelling Check (CSC) aims to identify and correct spelling errors in Chinese texts, where enhanced semantic understanding of a sentence can significantly improve correction accuracy. |
| Approach: | They propose a plug-and-play Alignment-and -Replacement module that enhances existing Chinese CSC models without retraining or fine-tuning. |
| Outcome: | The proposed module improves existing models while reducing retraining and fine-tuning. |
LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning (2026.acl-long)
Copied to clipboard
Zerui Chen, Qinggang Zhang, Zhishang Xiang, Zhimin Wei, Linfeng Gao, Xiao Huang, Zhihong Zhang, Jinsong Su
| Challenge: | Graph-based Retrieval-Augmented Generation (GraphRAG) is a new approach to document retrieval, but it is not suitable for legal reasoning. |
| Approach: | They propose a framework for reliable legal reasoning that structures knowledge as relational graphs and uses a multi-agent system to verify validity. |
| Outcome: | The proposed framework outperforms existing GraphRAG models in accurate and trustworthy legal analysis. |
Self-Evolution Fine-Tuning for Policy Optimization (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have showcased the remarkable capabilities and performance of large language models (LLMs) across a broad range of tasks. |
| Approach: | They propose supervised fine-tuning (SEFT) for LLM alignment to eliminate the need for annotated samples while retaining the stability and efficiency of SFT. |
| Outcome: | The proposed method eliminates the need for annotated samples while maintaining the stability and efficiency of SFT. |
TestAgent: An Adaptive and Intelligent Expert for Human Assessment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing adaptive testing methods face several challenges due to mechanized nature of most algorithms and noisy response data. |
| Approach: | They propose to use large language models to enhance adaptive testing through interactive engagement to capture test-takers’ responses and anomalies. |
| Outcome: | The proposed agent achieves more accurate results with 20% fewer questions than state-of-the-art baselines and testers preferred it in speed, smoothness, and other dimensions. |
Enhancing Neural Topic Model with Multi-Level Supervisions from Seed Words (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing topic seed words are difficult to incorporate into topic models due to the semantic diversity of natural language. |
| Approach: | They propose a neural topic model enhanced with supervisions from seed words on word and document levels. |
| Outcome: | The proposed model outperforms the state-of-the-art seeded topic models in terms of topic quality and classification accuracy. |
ARCH: Efficient Adversarial Regularized Training with Caching (2021.findings-emnlp)
Copied to clipboard
Simiao Zuo, Chen Liang, Haoming Jiang, Pengcheng He, Xiaodong Liu, Jianfeng Gao, Weizhu Chen, Tuo Zhao
| Challenge: | Existing approaches to regularize models require generating a perturbation for each sample in each epoch. |
| Approach: | They propose an adversarial regularization method where perturbations are generated and cached once every several epochs. |
| Outcome: | The proposed method significantly eases the computational burden (saves up to 70% of computational time) it produces a notably better (in most of the tasks) or comparable model generalization. |
Scaling up the State Size of RNN LLMs for Long-Context Scenarios (2025.acl-long)
Copied to clipboard
| Challenge: | Existing RNN-based LLMs struggle with long-context scenarios due to their quadratic computational complexity and linear memory requirements. |
| Approach: | They propose an efficient scaling method to scale RNN models to match the 2k context length of Transformers with small parameters overhead. |
| Outcome: | The proposed method improves long-context understanding and improves performance on FDA recall-intensive tasks. |
SelfAug: Mitigating Catastrophic Forgetting in Retrieval-Augmented Generation via Distribution Self-Alignment (2025.findings-emnlp)
Copied to clipboard
Yuqing Huang, Rongyang Zhang, Qimeng Wang, Chengqiang Lu, Yan Gao, null Yiwu, Yao Hu, Xuyang Zhi, Guiquan Liu, Xin Li, Hao Wang, Enhong Chen
| Challenge: | Existing solutions for supervised fine-tuning often lead to catastrophic forgetting, where models lose their previously acquired knowledge and general capabilities. |
| Approach: | They propose a self-distribution alignment method that aligns input sequence logits to preserve the model’s semantic distribution, thereby mitigating catastrophic forgetting and improving downstream performance. |
| Outcome: | The proposed method achieves a superior balance between downstream learning and general capability retention. |
CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models (2025.findings-naacl)
Copied to clipboard
Shangda Wu, Yashan Wang, Ruibin Yuan, Guo Zhancheng, Xu Tan, Ge Zhang, Monan Zhou, Jing Chen, Xuefeng Mu, Yuejie Gao, Yuanliang Dong, Jiafeng Liu, Xiaobing Li, Feng Yu, Maosong Sun
| Challenge: | Current music information retrieval systems struggle to meet linguistic diversity challenges . current systems struggle with text queries in non-English languages . |
| Approach: | They propose a music information retrieval system that supports both ABC notation and MIDI . CLaMP 2 includes a multilingual text encoder and a multiple-modal music encoder . |
| Outcome: | The proposed system achieves state-of-the-art results in multilingual semantic search and music classification across modalities. |
QFinZero: A Unified Financial Toolchain for LLM-Based Trading Agents (2026.acl-demo)
Copied to clipboard
Haochen Luo, Yifan LI, Ho Tin Ko, An Binh Minh, Junjie Xu, Tang Pok Hin, Wang Chak Wong, Gao Yuan, Zhengzhao Lai, Yuan Zhang, Chen Liu
| Challenge: | Existing trading systems rely on fragmented and task-specific APIs, resulting in inconsistent schemas and limited reproducibility. |
| Approach: | They propose a unified trading environment for large language model (LLM) agents that standardizes three core capabilities . they argue that such a standardized trading environment is essential for scalable research on LLM-based financial agents. |
| Outcome: | The proposed trading environment reduces engineering overhead and supports reproducible evaluation through comprehensive logging and deterministic replay. |
Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to medical text classification are struggling with imbalanced data distribution and rare labels. |
| Approach: | They propose a framework-agnostic algorithm that only utilizes internal label hierarchy in training deep learning models. |
| Outcome: | The proposed approach performs better on public datasets and real-world medical records than existing methods. |
When Efficiency Meets Safety: A Benchmark Security Analysis of KV Cache Compression in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Key-Value (KV) caching is widely used in large language models to enable long-context inference efficiently, yet its security implications remain underexplored. |
| Approach: | They propose a history-aware, per-head feedback merging strategy that prevents safety degradation while maintaining efficiency. |
| Outcome: | The proposed strategy prevents safety degradation while maintaining efficiency. |
PychoAgent: Psychology-driven LLM Agents for Explainable Panic Prediction on Social Media during Sudden Disaster Events (2025.emnlp-main)
Copied to clipboard
Mengzhu Liu, Zhengqiu Zhu, Chuan Ai, Chen Gao, Xinghong Li, Lingnan He, Kaisheng Lai, Yingfeng Chen, Xin Lu, Yong Li, Quanjun Yin
| Challenge: | Social media's rich information content and spatiotemporal granularity provide unique opportunities for emotion prediction and management. |
| Approach: | They propose a Psychology-driven generative Agent framework for explainable panic prediction based on emotion arousal theory. |
| Outcome: | The proposed framework improves panic emotion prediction performance by 13% to 21% compared to baseline models. |
MoDULA: Mixture of Domain-Specific and Universal LoRA for Multi-Task Learning (2024.emnlp-main)
Copied to clipboard
Yufei Ma, Zihan Liang, Huangyu Dai, Ben Chen, Dehong Gao, Zhuoran Ran, Wang Zihan, Linbo Jin, Wen Jiang, Guannan Zhang, Xiaoyan Cai, Libin Yang
| Challenge: | Recent advances in open-source Large Language Models (LLMs) have achieved notable successes in natural language processing. |
| Approach: | They propose a Parameter Efficient Fine-Tuning paradigm for improved fine-tuning and parameter efficiency in multi-task learning. |
| Outcome: | The proposed model outperforms existing methods on multi-task learning while reducing training costs by over 80% without losing general capability. |
DetectBench: Can Large Language Model Detect and Piece Together Implicit Evidence? (2024.findings-emnlp)
Copied to clipboard
Zhouhong Gu, Lin Zhang, Xiaoxuan Zhu, Jiangjie Chen, Wenhao Huang, Yikai Zhang, Shusen Wang, Zheyu Ye, Yan Gao, Hongwei Feng, Yanghua Xiao
| Challenge: | Existing LLMs' abilities to detect evidence in long contexts are far inferior to humans. |
| Approach: | They propose a benchmark to assess LLMs' abilities in evidence and multi-step commonsense reasoning within a long context. |
| Outcome: | The proposed method improves the performance of LLMs in evidence detection and commonsense reasoning. |
Thinking Economically: A Hierarchical Framework for Adaptive-Complexity Reasoning in LLMs (2026.findings-acl)
Copied to clipboard
Yubo Gao, Haotian Wu, Hong Chen, Junquan Huang, Yibo Yan, Jungang Li, Zihao Dongfang, Sicheng Tao, PS Tan, Jie Zhang, Xuming Hu
| Challenge: | Existing efficiency methods for Chain-of-Thought (CoT) generate excessively long rationales without commensurate accuracy gains. |
| Approach: | They propose a training framework that operationalizes this principle through coarse-to-fine budgeting. |
| Outcome: | Experiments on GSM8K and MATH500 show that HAB surpasses standard CoT in accuracy and reduces token usage, achieving stronger performance-efficiency trade-off than baselines. |
Exploring the Capability Boundaries of LLMs in Mastering of Chinese Chouxiang Language (2026.findings-acl)
Copied to clipboard
Dianqing Lin, Tian Lan, Jiali Zhu, Jiang Li, Wei Chen, Xu Liu, null Aruukhan, Xiangdong Su, Hongxu Hou, Guanglai Gao
| Challenge: | Current state-of-the-art LLMs exhibit clear limitations on multiple tasks, while performing well on tasks that involve contextual semantic understanding. |
| Approach: | They propose a mouse-based benchmark to evaluate LLMs' performance on NLP tasks involving Chouxiang Language. |
| Outcome: | The proposed benchmark evaluates the performance of LLMs on six NLP tasks involving Chouxiang Language. |
CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have found that information relevant to the next token prediction task accumulates in the hidden representations of just a few tokens. |
| Approach: | They propose a method that integrates attention preferences useful for a downstream task into the eviction process of hidden states. |
| Outcome: | The proposed method performs better on comprehension and retrieval tasks while preserving language modeling perplexity. |
CulFiT: A Fine-grained Cultural-aware LLM Training Paradigm via Multilingual Critique Data Synthesis (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models exhibit a specific cultural bias, neglecting values and differences of low-resource regions. |
| Approach: | They propose a culturally-aware training paradigm that leverages multilingual data and fine-grained reward modeling to enhance cultural sensitivity and inclusivity. |
| Outcome: | The proposed model achieves state-of-the-art in cultural alignment and general reasoning. |
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense (2025.naacl-long)
Copied to clipboard
Yang Ouyang, Hengrui Gu, Shuhang Lin, Wenyue Hua, Jie Peng, Bhavya Kailkhura, Meijun Gao, Tianlong Chen, Kaixiong Zhou
| Challenge: | Existing methods to defend against jailbreak attacks exploit vulnerabilities to elicit unintended or harmful outputs. |
| Approach: | They propose a method to defend against jailbreak attacks by patching specific layers within large language models through self-augmented datasets. |
| Outcome: | The proposed approach reduces harmfulness and attack success rate of jailbreak attacks without compromising utility for benign queries compared to previous methods. |
DiffCL: Difference-Aware Contrastive Learning for Automatic Answer Grading with Multi-Level Semantic Modeling (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to automate answer grading lack semantic understanding and scoring consistency. |
| Approach: | They propose a difference-aware AAG framework that integrates heuristic difference labeling with dual-contrastive learning. |
| Outcome: | The proposed method outperforms cross-entropy-based baselines on SciEntsBank and Beetle datasets. |
Dialogue Summarization with Static-Dynamic Structure Fusion Graph (2023.acl-long)
Copied to clipboard
| Challenge: | Dialogue summarization is a challenging task since it has dynamic interaction nature and inconsistent information flow among various speakers. |
| Approach: | They propose a Static-Dynamic graph-based Dialogue Summarization model which fuses prior knowledge from human expertise and adaptively learns the graph structure in an end-to-end learning fashion. |
| Outcome: | The proposed model can help people capture the highlights of a semi-structured and multi-participant dialogue without reviewing the complex dialogue context. |
Feedback to Reasoning: LLM-Assisted Molecular Optimization with Domain Feedback and Historical Reasoning (2026.findings-acl)
Copied to clipboard
Wenhan Gao, Xiran Fan, Chin-Chia Michael Yeh, Jiarui Sun, Yuzhong Chen, Menghai Pan, Mahashweta Das, Yi Liu
| Challenge: | Existing methods for molecular optimization do not leverage domain feedback and historical knowledge with reasoning traces and chemical insights. |
| Approach: | They propose a conversational molecular optimization pipeline that enables LLMs to accumulate and retrieve past actions, rationales, and feedback. |
| Outcome: | The proposed framework transforms LLMs from passive text generators into agentic experts that learn both actions and reasoning from experience. |
Mitigating Spurious Correlations in Text Classification Using Latent Space Geometry (2026.acl-long)
Copied to clipboard
| Challenge: | Existing models rely on predictive shortcuts that hold in training data but break under distribution shifts, leading to large performance drops for minority groups. |
| Approach: | They propose a framework that transforms abstract biases into interpretable geometric anchors without auxiliary classifiers by manipulating latent space geometry. |
| Outcome: | The proposed framework outperforms state-of-the-art baselines and improves worst-group accuracy by over 20% on the CivilComments dataset. |
Are Your LLMs Capable of Stable Reasoning? (2025.findings-acl)
Copied to clipboard
Junnan Liu, Hongwei Liu, Linchen Xiao, Ziyi Wang, Kuikun Liu, Songyang Gao, Wenwei Zhang, Songyang Zhang, Kai Chen
| Challenge: | Existing evaluation protocols and metrics do not capture the full spectrum of LLM capabilities, especially in complex reasoning tasks. |
| Approach: | They propose a new evaluation metric that continuously assesses model performance across multiple sampling attempts, quantifying both the model’s potential capabilities and operational consistency. |
| Outcome: | The proposed evaluation metric measures model performance across multiple sampling attempts and provides comprehensive insights into their potential capabilities and operational consistency. |
Measuring Meaning Composition in the Human Brain with Composition Scores from Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing computational metric to quantify extent of meaning composition is lacking . despite extensive neurolinguistic research, understanding how meaning is constructed is difficult . |
| Approach: | They propose a computational metric to quantify the degree of meaning composition . they use key-value memory interpretation of transformer feed-forward network blocks to study meaning composition. |
| Outcome: | Experimental results show that the metric correlates with brain clusters associated with word frequency, structural processing, and general sensitivity to words, suggesting multifaceted nature of meaning composition during human sentence comprehension. |
Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement (2026.acl-long)
Copied to clipboard
Zexu Sun, Yongcheng Zeng, Erxue Min, Heyang Gao, Bokai Ji, Dugang Liu, Xing Tang, Xiuqiang He, Xu Chen
| Challenge: | Recent advances in Large Language Models have demonstrated notable inferential capacities via reinforcement learning (RL) however, “zero-RL” approaches relying on fixed prompt templates introduce substantial sampling inefficiencies for weak LLMs. |
| Approach: | They propose a hierarchical metacognitive RL framework that decomposes zero-accuracy problems into subproblems and prompts the policy to refine answers by referencing previous wrong solutions. |
| Outcome: | The proposed framework improves sample utilization and sample efficiency and accelerates convergence compared to baselines. |
e5-omni: Explicit Cross-modal Alignment for Omni-modal Embeddings (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent omni-modal embeddings rely heavily on implicit alignment from pretrained visionlanguage models. |
| Approach: | They propose a lightweight explicit alignment recipe that adapts off-the-shelf VLMs into robust omni-modal embedding models. |
| Outcome: | The proposed model improves on MMEB-V2 and AudioCaps with a lightweight explicit alignment recipe. |
Rethinking Multiple-Choice Questions for RLVR: Unlocking Potential via Distractor Design (2026.findings-acl)
Copied to clipboard
Xu Guo, Qiming Ge, Jian Tong, Kedi Chen, Jin Zhang, Xiaogui Yang, Xuan Gao, Haijun Lv, Zhihui Lu, Yicheng Zou, Qipeng Guo
| Challenge: | Existing approaches to RLVR use multiple-choice questions as verifiable rewards . however, not all tasks provide reliable verification . |
| Approach: | They propose a framework that actively constructs high-quality distractors to block elimination shortcuts and promote deep reasoning. |
| Outcome: | The proposed method significantly improves reasoning capabilities of Large Language Models. |
Continually Improving Extractive QA via Human Feedback (2023.emnlp-main)
Copied to clipboard
| Challenge: | a study of extractive question answering systems using human feedback shows promising potential for continual learning. |
| Approach: | They study extractive question answering system by using user feedback to improve it . they design and deploy an iterative approach where users ask questions and provide feedback . |
| Outcome: | The proposed model improves over time across different data regimes and domains . human user feedback is more affordable and abundant than annotations provided by trained experts . |
What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can perform in-context learning (ICL) with only a few demonstrations, but its mechanisms are not well-understood. |
| Approach: | They characterize two ways in which LLMs leverage demonstrations to solve tasks with a few demonstrations. |
| Outcome: | The proposed model achieves non-trivial performance with only TR, and TR does not improve with larger models or more demonstrations. |
Holistic Sentence Embeddings for Better Out-of-Distribution Detection (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models have shown superior performance for textual OOD detection, but they estimate sample distance scores in the last-layer CLS embedding space. |
| Approach: | They propose to use token averaging and layer combination to boost OOD detection by deriving more holistic sentence embeddings. |
| Outcome: | The proposed method surpasses the state-of-the-art on a comprehensive suite of benchmarks by a 9.33% FAR95 margin. |
Toward Safe and Human-Aligned Game Conversational Recommendation via Multi-Agent Decomposition (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing systems for conversational recommender systems (CRS) have strong results in movies, but games present distinct challenges . MATCHA framework provides specialized agents for intent parsing, tool-augmented retrieval, multi-LLM ranking, and stronger safety. |
| Approach: | They propose a framework for conversational recommender systems that assigns specialized agents for intent parsing, tool-augmented retrieval, multi-LLM ranking and risk control. |
| Outcome: | MATCHA outperforms baselines on real user request dataset, improves Hit@5 by 20%, reduces popularity bias by 24%, and achieves 97.9% adversarial defense. |
Value type: the bridge to a better DST model (2023.findings-acl)
Copied to clipboard
| Challenge: | Value type of the slots can provide lots of useful information for DST tasks. however, it has been ignored in most previous works. |
| Approach: | They propose a new framework for DST task based on slot value type . they propose to extract the type of token from each turn and train a Ner model to extract corresponding type-entity from each conversation according to the token. |
| Outcome: | The proposed framework is effective on two multi-domain task-oriented conversation datasets. |
TransCoder: Towards Unified Transferable Code Representation Learning Inspired by Human Skills (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods to fine-tune code intelligence models to individual tasks are costly and require large data sets. |
| Approach: | They propose a Transferable fine-tuning strategy for Code representation learning that uses a tunable prefix encoder to capture cross-task and cross-language transferable knowledge and apply it to downstream adaptation. |
| Outcome: | The proposed method can lead to superior performance on code-related tasks and encourage mutual reinforcement. |
UBench: Benchmarking Uncertainty in Large Language Models with Multiple Choice Questions (2025.findings-acl)
Copied to clipboard
Xunzhi Wang, Zhuowei Zhang, Gaonan Chen, Qiongyu Li, Bitong Luo, Zhixin Han, Haotian Wang, Zhiyu Li, Hang Gao, Mengting Hu
| Challenge: | Existing methods for benchmarking the uncertainty of large language models face challenges . existing methods require internal model access, additional training, or high computational costs . |
| Approach: | They propose a new benchmark for evaluating the uncertainty of large language models based on confidence intervals . UBench encompasses 11,978 multiple choice questions spanning knowledge, language, understanding, and reasoning capabilities. |
| Outcome: | The proposed method outperforms existing methods for benchmarking the uncertainty of large language models. |
HittER: Hierarchical Transformers for Knowledge Graph Embeddings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding methods to learn representations of knowledge graphs are conceptually simple and can be applied to tasks like factoid question answering (Saxena et al., 2020) and reasoning. |
| Approach: | They propose a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity’s neighborhood. |
| Outcome: | The proposed model achieves state-of-the-art on multiple link prediction datasets and can be integrated into BERT and demonstrate its effectiveness on two Freebase factoid question answering datasets. |
Discriminative Deep Dyna-Q: Robust Planning for Dialogue Policy Learning (D18-1)
Copied to clipboard
| Challenge: | Existing approaches to improve the effectiveness and robustness of Deep Dyna-Q (DDQ) are based on a discriminator to control the quality of simulated experiences and to improve learning. |
| Approach: | They propose to use an RNN-based discriminator to control the quality of simulated experience to improve the effectiveness and robustness of Deep Dyna-Q. |
| Outcome: | The proposed framework outperforms DDQ by controlling the quality of simulated experience used for planning. |
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments (2026.acl-long)
Copied to clipboard
Haotian Xu, Yue Hu, Zhengqiu Zhu, Chen Gao, Ziyou Wang, Junreng Rao, Wenhao Lu, Weishi Li, Quanjun Yin, Yong Li
| Challenge: | Existing benchmarks focus on indoor or street settings, overlooking challenges of open-ended urban spaces. |
| Approach: | They propose a benchmark to probe cross-view spatial reasoning capabilities of current VLMs in urban settings. |
| Outcome: | The citycube benchmark examines the performance of current vision-language models in urban environments. |
ALPS: Attention Localization and Pruning Strategy for Efficient Adaptation of Large Language Models (2025.findings-acl)
Copied to clipboard
Hao Chen, Haoze Li, Zhiqing Xiao, Lirong Gao, Qi Zhang, Xiaomeng Hu, Ningtao Wang, Xing Fu, Junbo Zhao
| Challenge: | Prior research has focused on optimizing general-purpose large language models to downstream tasks . however, these approaches inherently introduce data dependency, which hinders generalization and reusability. |
| Approach: | They propose an algorithm that localizes the most task-sensitive attention heads and prunes by restricting attention training updates to these heads, thereby reducing alignment costs. |
| Outcome: | The proposed algorithm achieves 2% performance improvement over baselines on three tasks while localizing the most task-sensitive attention heads. |
Live-Aid: A Large-Scale Dialogue Dataset and Benchmark for Interleaved Multi-party Interactions in Live Streaming (2026.findings-acl)
Copied to clipboard
Yiming Lei, Yize Fan, Zeming Liu, Jiaji Dong, Hui Qiu, Haitao Leng, Qingjie Liu, Kehai Chen, Tingting Gao, Yunhong Wang
| Challenge: | Existing Multimodal Large Language Models struggle with dynamic interactions due to the scarcity of high-quality interleaved data. |
| Approach: | They propose a large-scale interleaved live interaction Chinese dataset with human-annotated video responses. |
| Outcome: | The proposed model can be used to evaluate live interactions in Chinese over 1,100 hours and 80,037 dialogue turns. |
SciVerse: Unveiling the Knowledge Comprehension and Visual Reasoning of LMMs on Multi-modal Scientific Problems (2025.findings-acl)
Copied to clipboard
| Challenge: | SciVerse is a multi-modal scientific evaluation benchmark to assess large multi-models . it examines the scientific knowledge comprehension, multi-mod content interpretation and Chain-of-Thought reasoning . authors examine the scientific proficiency of LMMs in scientific domains based on their work . |
| Approach: | They propose a multi-modal scientific evaluation benchmark to thoroughly assess Large Multi-modal Models across 5,735 test instances in five different versions. |
| Outcome: | The proposed evaluation reveals critical limitations in LMMs' scientific proficiency and provides new insights into future developments. |
How to Train Long-Context Language Models (Effectively) (2025.acl-long)
Copied to clipboard
| Challenge: | a new study shows that language models can process extremely long contexts with minimal training. |
| Approach: | They use supervised fine-tuning and continued training to evaluate a language model's long-context capabilities. |
| Outcome: | The proposed model outperforms Llama-3.1-8B-Instruct on most long-context tasks . the model can process 512K tokens, one of the longest context windows of LMs . |
MoLA: MoE LoRA with Layer-wise Expert Allocation (2025.findings-naacl)
Copied to clipboard
Chongyang Gao, Kezhen Chen, Jinmeng Rao, Ruibo Liu, Baochen Sun, Yawen Zhang, Daiyi Peng, Xiaoyuan Guo, Vs Subrahmanian
| Challenge: | Recent efforts to integrate low-rank adaptation (LoRA) with the Mixture-of-Experts (MoE) have achieved performance comparable to full-parameter fine-tuning by tuning much fewer parameters. |
| Approach: | They propose a parameter-efficient MoE method for low-rank adaptation with the Mixture-of-Experts (MoE) they use layers of LoRA experts to allocate more LoRA expert to middle layers . |
| Outcome: | The proposed method outperforms baseline models on six well-known NLP and commonsense QA benchmarks on LLAMA-2, Mistral, and Gemma. |
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to elicit confidence from large language models are limited to binary or inaccurate group-level confidence estimates. |
| Approach: | They propose a training framework that teaches LLMs to express more fine-grained confidence estimates. |
| Outcome: | The proposed training framework reduces the confidence calibration error and maintains the performance of the model. |
Huatuo-26M, a Large-scale Chinese Medical QA Dataset (2025.findings-naacl)
Copied to clipboard
Xidong Wang, Jianquan Li, Shunian Chen, Yuxuan Zhu, Xiangbo Wu, Zhiyi Zhang, Xiaolong Xu, Junying Chen, Jie Fu, Xiang Wan, Anningzhe Gao, Benyou Wang
| Challenge: | Large Language Models are a powerful tool for medical research, but the data is a bottleneck. |
| Approach: | They propose to use the largest ever medical Question Answering dataset with 26 Million QA pairs as a fine-tuning data for training large language models. |
| Outcome: | The proposed dataset demonstrates that it can be used to train large language models and improves zero-shot performance on other datasets. |
Realistic Training Data Generation and Rule Enhanced Decoding in LLM for NameGuess (2025.emnlp-main)
Copied to clipboard
| Challenge: | Abbreviated column names often harm downstream tasks, causing performance drops of 10.54, 40.50, and 3.83 percentage points. |
| Approach: | They propose a method that integrates a subsequence abbreviation generator trained on human-annotated data and collects non-subsequent abbrevations to improve the training set. |
| Outcome: | The proposed approach improves on the English NameGuess task and surpasses state-of-the-art LLMs. |
Iterative Constrained Back-Translation for Unsupervised Domain Adaptation of Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Existing back-translation methods focus on in-domain lexical knowledge, which may lead to poor translation of unseen in- domain words. |
| Approach: | They propose an iterative constrained back-translation method to incorporate in-domain lexical knowledge into synthetic parallel data from BT. |
| Outcome: | The proposed method improves the BLEU score by up to 3.08 on four domains. |
Open Set Relation Extraction via Unknown-Aware Training (2023.acl-long)
Copied to clipboard
Jun Zhao, Xin Zhao, WenYu Zhan, Qi Zhang, Tao Gui, Zhongyu Wei, Yun Wen Chen, Xiang Gao, Xuanjing Huang
| Challenge: | Existing supervised relation extraction methods can still misclassify unknown relations into known relations due to the lack of supervision signals. |
| Approach: | They propose a method that regularizes the model by dynamically synthesizing negative instances that can provide the missing supervision signals. |
| Outcome: | The proposed method achieves SOTA unknown relation detection without compromising the classification of known relations. |
MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria (2025.naacl-long)
Copied to clipboard
Wentao Ge, Shunian Chen, Hardy Chen, Nuo Chen, Junying Chen, Zhihong Chen, Wenya Xie, Shuo Yan, ChenghaoZhu ChenghaoZhu, Ziyue Lin, Dingjie Song, Xidong Wang, Anningzhe Gao, Zhang Zhiyi, Jianquan Li, Xiang Wan, Benyou Wang
| Challenge: | Existing evaluation methodologies for multimodal large language models are limited in evaluating objective queries without considering real-world user experiences. |
| Approach: | They propose to evaluate multimodal large language models with per-sample criteria using potent MLLM as the judge. |
| Outcome: | The proposed evaluation paradigm shows that it can be used to evaluate multimodal large language models with per-sample criteria. |
MoDification: Mixture of Depths Made Easy (2025.naacl-long)
Copied to clipboard
Chen Zhang, Meizhi Zhong, Qimeng Wang, Xuantao Lu, Zheyu Ye, Chengqiang Lu, Yan Gao, Yao Hu, Kehai Chen, Min Zhang, Dawei Song
| Challenge: | Long-context efficiency is a trending topic in large language model (LLM) serving. |
| Approach: | They propose a method to combine long-context efficiency and mixture of depths to bring down both latency and memory. |
| Outcome: | The proposed method achieves 1.2 speedup in latency and 1.8 reduction in memory compared to original LLMs especially in long-context applications. |
When Raw Data Prevails: Are Large Language Model Embeddings Effective in Numerical Data Representation for Medical Machine Learning Applications? (2024.findings-emnlp)
Copied to clipboard
Yanjun Gao, Skatje Myers, Shan Chen, Dmitriy Dligach, Timothy Miller, Danielle Bitterman, Matthew Churpek, Majid Afshar
| Challenge: | Numerical data is pivotal for medical questions and answers, but tabular data is not fully integrated into LLMs. |
| Approach: | They examine the effectiveness of vector representations from last hidden states of LLMs for medical diagnostics and prognostics using electronic health record data. |
| Outcome: | The proposed representations outperform those using raw numerical EHR data in medical diagnostics and prognostics. |
Agentic Conversational Search with Contextualized Reasoning via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
Fengran Mo, Yifan Gao, Sha Li, Hansi Zeng, Xin Liu, Zhaoxuan Tan, Xian Li, Jianshu Chen, Dakuo Wang, Meng Jiang
| Challenge: | Existing studies focus on single-turn scenarios, which might lack the ability to handle multi-turn interactions. |
| Approach: | They propose a conversational agent that interleaves search and reasoning across turns and provides tailored rewards towards evolving user goals. |
| Outcome: | The proposed agent interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training with tailored rewards towards evolving user goals. |
DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods that only address a fixed set of fields are difficult to use for different form types. |
| Approach: | They propose a value retrieval method with arbitrary queries for form-like documents . they propose 'docQueryNet' to predict target value based on understanding of layout and semantics of a form . |
| Outcome: | The proposed method outperforms existing methods on value retrieval . it improves document understanding on large-scale model pre-training by 17% . |
Revisiting LoRA through the Lens of Parameter Redundancy: Spectral Encoding Helps (2025.findings-acl)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) has emerged as a prominent technique for fine-tuning large foundation models. |
| Approach: | They propose a low-rank Adaptation technique that harnesses the expressiveness of spectral bases to re-parameterize LoRA from a sparse spectral subspace. |
| Outcome: | The proposed technique achieves greater efficiency with fewer parameters than baselines on various downstream tasks, including commonsense reasoning, math reasoning, and code generation. |
The Right Time Matters: Data Arrangement Affects Zero-Shot Generalization in Instruction Tuning (2025.findings-acl)
Copied to clipboard
Bingxiang He, Ning Ding, Cheng Qian, Jia Deng, Ganqu Cui, Lifan Yuan, Haiwen Hong, Huan-ang Gao, Longtao Huang, Hui Xue, Huimin Chen, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing work on instruction tuning has focused on task level, without considering that tasks are artificially defined and, to LLMs, merely consist of tokens and representations. |
| Approach: | They propose a training data arrangement framework that allows for continual learning and loss reduction. |
| Outcome: | The proposed framework promotes continual learning and loss reduction on unseen tasks. |
UrbanLLM: Autonomous Urban Activity Planning and Management with Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | UrbanLLM is a fine-tuned large language model designed to tackle diverse urban problems. |
| Approach: | They propose a fine-tuned large language model to tackle diverse urban problems . UrbanLLM decomposes urban-related queries into manageable sub-tasks . |
| Outcome: | The proposed model outperforms existing models in urban planning and management tasks. |
MDTeamGPT: Mitigating Context Collapse and Enabling Self-Evolution in Medical Multi-Agent Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Long, multi-round, multirole interaction trajectories lead to severe information dilution and context window overload, triggering context collapse which destabilizes reasoning. |
| Approach: | They propose a multi-agent framework that compresses and reorganizes multi-round consensus. |
| Outcome: | The proposed framework outperforms baselines across text-based and multimodal tasks while demonstrating superior diagnostic performance and stability in complex clinical scenarios. |
Uncertainty-aware Parameter-Efficient Self-training for Semi-supervised Language Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for pre-trained language models rely on noisy data, which can be expensive if all parameters are updated. |
| Approach: | They propose a self-training framework that incorporates Monte Carlo dropouts into the model and judiciously selects reliable pseudo-labeled examples based on confidence and certainty. |
| Outcome: | The proposed framework improves performance and efficiency over multiple tasks over multiple datasets. |
Understanding the RoPE Extensions of Long-Context LLMs: An Attention Perspective (2025.coling-main)
Copied to clipboard
| Challenge: | Enabling LLMs to handle lengthy context is currently a research hotspot . a notable challenge limiting further customization is the inability of LLM to utilize context beyond pretrained length due to the inherent flaw of rotary position embedding (RoPE). |
| Approach: | They propose to extend the RoPE from an attention perspective and on two benchmarking tasks. |
| Outcome: | The proposed extension of the RoPE improves extrapolation and retrieval errors. |
CycleAlign: Iterative Distillation from Black-box LLM to White-box Models for Better Human Alignment (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing language models that generate harmful responses are constrained by their inherent capability. |
| Approach: | They propose to align large language models with human preferences from AI feedback. |
| Outcome: | The proposed framework improves the alignment of large language models with human preferences from AI feedback. |
An Efficient Context-Dependent Memory Framework for LLM-Centric Agents (2025.naacl-industry)
Copied to clipboard
| Challenge: | a recent study has demonstrated that context-dependent memory encoding can help to retrieve key memory cues essential for problem-solving. |
| Approach: | They propose an efficient architecture miming human memory processes through multistage encoding, context-aware storage, and retrieval strategies for LLM-centric agents. |
| Outcome: | The proposed architecture surpasses state-of-the-art online LLM-centric approaches on two interactive decision-making benchmarks in the navigation and manipulation domain. |
IGenBench: Benchmarking the Reliability of Text-to-Infographic Generation (2026.acl-long)
Copied to clipboard
Yinghao Tang, Xueding Liu, Boyuan Zhang, Tingfeng Lan, Yupeng Xie, Jiale Lao, Yiyao Wang, Haoxuan Li, Tingting Gao, Bo Pan, Luoxuan Weng, Xiuqi Huang, Minfeng Zhu, Yingchaojie Feng, Yuyu Luo, Wei Chen
| Challenge: | Generated infographics may appear correct at first glance but contain easily overlooked issues, such as distorted data encoding or incorrect textual content. |
| Approach: | They propose to evaluate reliability of text-to-infographic generation using IGenBench . they employ multimodal large language models to verify each question . |
| Outcome: | The proposed framework decomposes reliability verification into atomic yes/no questions based on a taxonomy of 10 question types. |
MultiFinBen: Benchmarking Large Language Models for Multilingual and Multimodal Financial Application (2026.acl-long)
Copied to clipboard
Xueqing Peng, Lingfei Qian, Yan Wang, Ruoyu Xiang, Yueru He, Yang Ren, Mingyang Jiang, Vincent Jim Zhang, Yuqing Guo, Jeff Zhao, Huan He, Yi Han, Yun Feng, Yuechen Jiang, Yupeng Cao, Haohang Li, Yangyang Yu, Xiaoyu Wang, Penglei Gao, Shengyuan Lin, Keyi Wang, Shanshan Yang, Yilun Zhao, Zhiwei Liu, Peng Lu, Jerry Huang, Suyuchen Wang, Triantafillos Papadopoulos, Polydoros Giannouris, Efstathia Soufleri, Nuo Chen, Zhiyang Deng, Heming Fu, Yijia Zhao, Mingquan Lin, Meikang Qiu, Kaleb E Smith, Arman Cohan, Xiao-Yang Liu, Jimin Huang, Guojun Xiong, Alejandro Lopez-Lira, Xi Chen, Junichi Tsujii, Jian-Yun Nie, Sophia Ananiadou, Qianqian Xie
| Challenge: | Existing evaluations of LLMs in finance are text-only, monolingual, and largely saturated by current models. |
| Approach: | They propose a multilingual and multimodal benchmark for evaluating LLMs in real financial contexts. |
| Outcome: | The first expert-annotated multilingual and multimodal benchmark is released . it evaluates 21 leading LLMs and shows they perform better in multilingual settings . |
Learning to Maximize Mutual Information for Chain-of-Thought Distillation (2024.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation is a technique of transferring knowledge from large, complex models to smaller ones. |
| Approach: | They propose a method utilizing chain-of-thought distillation to transfer knowledge from large, complex models to smaller ones by maximizing mutual information of the representation features of the two tasks. |
| Outcome: | The proposed method outperforms the state-of-the-art knowledge distillation method on four datasets. |
Beyond Single-Audio: Advancing Multi-Audio Processing in Audio Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evaluations of audio large language models focus on single audio inputs, but real-world applications often require processing multiple audio streams simultaneously. |
| Approach: | They propose a multi-audio evaluation benchmark that combines 20 audio inputs from 11 audio tasks to capture audio context. |
| Outcome: | The proposed model outperforms baseline models and achieves high data efficiency without human annotations. |
Open-Set Living Need Prediction with Large Language Models (2025.findings-acl)
Copied to clipboard
Xiaochong Lan, Jie Feng, Yizhou Sun, Chen Gao, Jiahuan Lei, Xinleishi Xinleishi, Hengliang Luo, Yong Li
| Challenge: | Existing approaches to living need prediction treat it as a closed-set classification problem, severely limiting their ability to capture diversity and complexity of living needs. |
| Approach: | They propose a system leveraging large language models for unrestricted need prediction that leverages Maslow's hierarchy of needs to align predictions with human living needs. |
| Outcome: | The proposed system outperforms closed-set approaches on need-based life service recall by an average of 19.37% on real-world datasets. |
STEP: Success-Rate-Aware Trajectory-Efficient Policy Optimization (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing GRPO-based methods allocate sampling uniformly across tasks regardless of difficulty, propagate misleading learning signals and incur high sample-collection costs. |
| Approach: | They propose a framework that allocates sampling based on per-task success rates and performs fine-grained step-level optimization. |
| Outcome: | The proposed method improves sample efficiency and training stability over existing GRPO variants and three ablation variants on OSWorld and AndroidWorld. |
QiMeng-Attention: SOTA Attention Operator is generated by SOTA Attention Algorithm (2025.findings-acl)
Copied to clipboard
Qirui Zhou, Shaohui Peng, Weiqiang Xiong, Haixin Chen, Yuanbo Wen, Haochen Li, Ling Li, Qi Guo, Yongwei Zhao, Ke Gao, Ruizhi Chen, Yanjun Wu, Zhao Chen, Yunji Chen
| Challenge: | Existing LLMs cannot comprehend the complex data flow and computation process of the attention operator and utilize low-level primitive to exploit GPU performance. |
| Approach: | They propose an LLM-friendly Thinking Language (LLM-TL) that can decouple the generation of high-level optimization logic and low-level implementation on GPU and enhance LLMs’ understanding of attention operator. |
| Outcome: | The proposed method outshines existing LLMs on A100, RTX8000, and T4 GPUs, achieving a speed-up of up to 35.16. |
An Exploratory Study on Model Compression for Text-to-SQL (2023.findings-acl)
Copied to clipboard
| Challenge: | Text-to-SQL translates user queries into SQL statements that can retrieve relevant answers from relational databases. |
| Approach: | They propose to apply model compression techniques to sketch-based and sequence-to-sequence Text-toSQL models. |
| Outcome: | The proposed models have higher inference efficiency and respond better to model compression than sequence-to-sequence models. |
Learning “O” Helps for Learning More: Handling the Unlabeled Entity Problem for Class-incremental NER (2023.acl-long)
Copied to clipboard
Ruotian Ma, Xuanting Chen, Zhang Lin, Xin Zhou, Junzhe Wang, Tao Gui, Qi Zhang, Xiang Gao, Yun Wen Chen
| Challenge: | Existing Named Entity Recognition systems are typically trained on a large-scale dataset with predefined entity classes, then deployed for entity recognition on the test data without further adaptation or refinement. |
| Approach: | They propose a representation learning method that adaptively detects entity clusters in "O" and two effective distance-based relabeling strategies for better learning the old classes. |
| Outcome: | The proposed method achieves 10.62% improvement over the baseline methods. |
UnitedQA: A Hybrid Approach for Open Domain Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Recent work on open-domain question answering focuses on either extractive or generative readers exclusively. |
| Approach: | They propose a hybrid approach to extractive and generative readers that leverages both models. |
| Outcome: | The proposed approach outperforms state-of-the-art models on NaturalQuestions and TriviaQA respectively. |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
ReAgent: Reversible Multi-Agent Reasoning for Knowledge-Enhanced Multi-Hop QA (2025.emnlp-main)
Copied to clipboard
Zhao Xinjie, Fan Gao, Xingyu Song, Yingjian Chen, Rui Yang, Yanran Fu, Yuyang Wang, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Multi-hop question answering (QA) is a central challenge in natural language processing . early mistakes can cause errors and undermine the final result, authors say . |
| Approach: | They propose a reversible multi-agent reasoning framework that backtracks to earlier valid states when conflicts arise. |
| Outcome: | Empirical evaluation shows that the framework improves on forward-only benchmarks by 6% . the approach enables agents to backtrack to valid states when conflicts arise . |
Multilingual Pretraining and Instruction Tuning Improve Cross-Lingual Knowledge Alignment, But Only Shallowly (2024.naacl-long)
Copied to clipboard
| Challenge: | Current large language models show imbalance abilities in different languages . authors propose two approaches to improve cross-lingual knowledge alignment . |
| Approach: | They propose a framework to assess cross-lingual knowledge alignment of large language models . they propose multilingual pretraining and multilingual instruction tuning to address this problem . |
| Outcome: | The proposed framework assesses the cross-lingual knowledge alignment of LLMs in performance, consistency and conductivity levels. |
Implicit Deep Latent Variable Models for Text Generation (D19-1)
Copied to clipboard
| Challenge: | Variational auto-encoders have been used for text generation but their representation power is limited due to two reasons. |
| Approach: | They advocate sample-based representations of variational distributions for natural language . they further develop an LVM to directly match the aggregated posterior to the prior . |
| Outcome: | The proposed model can be viewed as a natural extension of VAEs with a regularization of maximizing mutual information, mitigating the "posterior collapse" issue. |
User Feedback Alignment for LLM-powered Exploration in Large-scale Recommendation Systems (2025.acl-industry)
Copied to clipboard
Jianling Wang, Yifan Liu, Yinghao Sun, Xuejian Ma, Yueqi Wang, He Ma, Zhengyang Su, Minmin Chen, Mingyan Gao, Onkar Dalal, Ed H. Chi, Lichan Hong, Ningren Han, Haokai Lu
| Challenge: | Large Language Models (LLMs) can be used to broaden user experiences beyond established preferences and reinforce feedback loops. |
| Approach: | They propose a hierarchical approach that combines hierarchic planning with LLM inference-time scaling to improve recommendation relevancy without compromising novelty. |
| Outcome: | The proposed approach shows significant gains in both user satisfaction and exploration diversity. |
Do Current Video LLMs Have Strong OCR Abilities? A Preliminary Study (2025.coling-main)
Copied to clipboard
| Challenge: | a new benchmark evaluates video-based optical character recognition (Video OCR) performance of multi-modal models in videos . the benchmark aims to improve video LLMs' ability to extract text from video content . previous benchmarks have focused on video QA, but not video-related QA. |
| Approach: | They propose to evaluate the video OCR performance of multi-modal models in videos . they use a semi-automated approach that integrates the OCR ability of image LLMs with manual refinement . |
| Outcome: | The proposed benchmark includes 1,028 videos and 2,961 question-answer pairs . it integrates the OCR ability of image LLMs with manual refinement . |
Uncovering and Categorizing Social Biases in Text-to-SQL (2023.acl-long)
Copied to clipboard
| Challenge: | Existing Text-to-SQL models are trained on clean, neutral datasets, such as Spider and WikiSQl, but these models contain social bias at different rates. |
| Approach: | They propose to use data to map natural language utterances to SQL queries. |
| Outcome: | The proposed model can contain social bias at different rates in the downstream Text-to-SQL task. |
FinMRAGBench: A Realistic and Complex Benchmark for Multi-Modal RAG in Financial Document Analysis (2026.findings-acl)
Copied to clipboard
Shouqing Yang, Qi Zhang, Yuhang Yang, Ruikang Xu, Yuwei Hou, Zhulin Jia, Lirong Gao, Haobo Wang, Jinglei Chen, Jiexiang Wang, Sheng Guo, Bo Zheng, Gang Chen
| Challenge: | Existing benchmarks for realistic financial analysis fail to capture realistic financial situations involving cross-document retrieval, multi-page evidence integration, and diverse analytical tasks. |
| Approach: | They propose a multi-modal financial RAG benchmark that evaluates large language models in realistic financial analysis settings. |
| Outcome: | The proposed framework achieves the strongest overall performance across all models. |
Enhancing Scalability of Pre-trained Language Models via Efficient Parameter Sharing (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to scale pre-trained language models to a deeper model depth share all parameters or use extra blocks. |
| Approach: | They propose a parameter-efficient approach to scaling pre-trained language models to a deeper model depth using matrix product operator. |
| Outcome: | The proposed model scales pre-trained language models to a deeper model depth by 4x and achieves 0.1 points higher than BERT-large for GLUE score. |
AMPO: Automatic Multi-Branched Prompt Optimization (2024.emnlp-main)
Copied to clipboard
Sheng Yang, Yurong Wu, Yan Gao, Zineng Zhou, Bin Zhu, Xiaodi Sun, Jian-Guang Lou, Zhiming Ding, Anbang Hu, Yuan Fang, Yunsong Li, Junyan Chen, Linjun Yang
| Challenge: | Existing prompt engineering techniques are limited to producing single flow instructions, struggling with handling diverse patterns. |
| Approach: | They propose an automatic prompt optimization method that iteratively develops a multi-branched prompt using failure cases as feedback. |
| Outcome: | The proposed method achieves the best results across five tasks and demonstrates significant optimization efficiency due to adoption of a minimal search strategy. |
Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have shown strong abilities to understand natural language, but how these factors affect the models’ language perception is unclear. |
| Approach: | They compare the self-attention of several existing large language models in different sizes to assess the effect of scaling and instruction tuning on language perception. |
| Outcome: | The proposed models are closer to non-native speakers than native speakers in attention, suggesting a sub-optimal language perception of all models. |
Aligning Large Language Models to Follow Instructions and Hallucinate Less via Effective Data Filtering (2025.acl-long)
Copied to clipboard
Shuzheng Si, Haozhe Zhao, Gang Chen, Cheng Gao, Yuzhuo Bai, Zhitong Wang, Kaikai An, Kangyang Luo, Chen Qian, Fanchao Qi, Baobao Chang, Maosong Sun
| Challenge: | Existing studies show that training LLMs on data containing unfamiliar knowledge during instruction tuning can encourage hallucinations. |
| Approach: | They propose a framework that measures how familiar the LLM is with instruction data and introduce an expert-aligned reward model to ensure the quality of selected samples. |
| Outcome: | The proposed framework reduces hallucinations while maintaining a competitive ability to follow instructions. |
Ditch the Gold Standard: Re-evaluating Conversational Question Answering (2022.acl-long)
Copied to clipboard
| Challenge: | Existing conversational question answering systems provide natural-language answers to users in information-seeking conversations. |
| Approach: | They conduct the first large-scale human evaluation of state-of-the-art conversational question answering systems . they propose a question rewriting mechanism based on predicted history which better correlates with human judgments . |
| Outcome: | The proposed question rewriting mechanism better correlates with human judgments. |
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)
Copied to clipboard
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces (2025.acl-long)
Copied to clipboard
Baining Zhao, Jianjie Fang, Zichao Dai, Ziyou Wang, Jirong Zha, Weichen Zhang, Chen Gao, Yue Wang, Jinqiang Cui, Xinlei Chen, Yong Li
| Challenge: | Large multimodal models exhibit remarkable intelligence, yet their embodied cognitive abilities during motion in open-ended urban aerial spaces remain to be explored. |
| Approach: | They propose a benchmark to evaluate whether large multimodal models can process continuous first-person visual observations like humans. |
| Outcome: | The proposed model can process first-person visual observations like humans, enabling recall, perception, reasoning, and navigation. |
Improving the Robustness of Summarization Systems with Dual Augmentation (2023.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that state-of-the-art summarization models have a significant decrease in performance on adversarial and noisy test sets. |
| Approach: | They propose a SummAttacker approach to generate adversarial samples based on pre-trained language models that can generate word-level synonym substitution and noise. |
| Outcome: | The proposed model performs better on noisy, attacked, and clean datasets than baseline models and is more robust on noisy and attacked datasets. |
LLM-as-a-Coauthor: Can Mixed Human-Written and Machine-Generated Text Be Detected? (2024.findings-naacl)
Copied to clipboard
Qihui Zhang, Chujie Gao, Dongping Chen, Yue Huang, Yixin Huang, Zhenyang Sun, Shilin Zhang, Weiye Li, Zhengyan Fu, Yao Wan, Lichao Sun
| Challenge: | Current research focuses on purely MGT detection without adequately addressing mixed scenarios including AI-revised Human-Written Text (HWT) and human-revealed MGT. |
| Approach: | They define mixtext, a form of mixed text involving both AI and human-generated content, and then use a MixSet dataset to assess their effectiveness. |
| Outcome: | The proposed detectors struggle to identify mixtext, particularly in dealing with subtle modifications and style adaptability. |
Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning (2023.findings-emnlp)
Copied to clipboard
Yougang Lyu, Jitai Hao, Zihan Wang, Kai Zhao, Shen Gao, Pengjie Ren, Zhumin Chen, Fang Wang, Zhaochun Ren
| Challenge: | Existing methods for predicting judgment results for multiple defendants are ineffective. |
| Approach: | They propose a method to predict the judgment results for each defendant in multi-defendant cases . they formalize the multi-diffendant judgment process as hierarchical reasoning chains . |
| Outcome: | The proposed method can predict the judgment results for multiple defendants in multi-defendant cases. |
Weighted Contrastive Learning With False Negative Control to Help Long-tailed Product Classification (2023.acl-industry)
Copied to clipboard
| Challenge: | Item categorization (IC) aims to classify a product into leaf nodes in a categorical taxonomy due to scarce supervision. |
| Approach: | They propose to use K-positive contrastive loss (KCL) to address IC task’s long-tail issue by re-weighting positive pairs in the KCL loss with a regularization that the sum of weights should be constrained to K+1 as close as possible. |
| Outcome: | The proposed method improves on the long-tail issue in the image classification task and when using text-based contrastive learning, it can be applied on the IC task. |
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)
Copied to clipboard
Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song
| Challenge: | Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges . |
| Approach: | They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions. |
| Outcome: | The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset . |
MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environments (2026.acl-long)
Copied to clipboard
Quyu Kong, Xu Zhang, Zhenyu Yang, Nolan Gao, Chen Liu, Panrong Tong, Chenglin Cai, Hanzhang Zhou, Jianan Zhang, Liangyu Chen, Zhidan Liu, Steven Hoi, Yue Wang
| Challenge: | AndroidWorld is the dominant mobile GUI agent evaluation benchmark, but its success rates are low . despite reproducible emulator environment, it lacks key application categories such as e-commerce and enterprise communication. |
| Approach: | They propose a benchmark for mobile GUI agents that reflects real-world usage through long-horizon, cross-application workflows. |
| Outcome: | The proposed framework achieves over 90% success rates, while AndroidWorld is the dominant benchmark. |
When Personalization Tricks Detectors: The Feature-Inversion Trap in Machine-Generated Text Detection (2026.acl-long)
Copied to clipboard
Lang Gao, Xuhui Li, Chenxi Wang, Mingzhe Li, Wei Liu, Zirui Song, Jinghui Zhang, Rui Yan, Preslav Nakov, Xiuying Chen
| Challenge: | Personalized MGT detection remains largely underexplored due to personalization challenges . large language models (LLMs) can imitate personal writing styles, but they can generate fake news and misinformation. |
| Approach: | They propose a benchmark to evaluate detector robustness under personalization . they attribute this limitation to a feature-inversion trap that flips the effect in personalized contexts . |
| Outcome: | The proposed framework predicts detector robustness under personalization with an 85% correlation to actual results. |
Generation-Augmented Retrieval for Open-Domain Question Answering (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to answer open-domain questions use sparse representations and sparsity. |
| Approach: | They propose a method which augments a query by generating relevant contexts from heuristically discovered contexts without external supervision. |
| Outcome: | The proposed approach outperforms state-of-the-art dense retrieval methods on natural questions and triviaQA datasets. |
Enhancing Legal Case Retrieval via Scaling High-quality Synthetic Query-Candidate Pairs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies focus on case-to-case retrieval using lengthy queries, which does not match real-world scenarios. |
| Approach: | They propose a method to construct query-candidate pairs and build the largest LCR dataset to date, LEAD. |
| Outcome: | Experimental results show that the method can provide ample training signals for LCR models. |
Noisy-Labeled NER with Confidence Estimation (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies in deep learning have shown significant progress in named entity recognition (NER) . however, most existing works assume clean data annotation, while real-world data typically involve a large amount of noises. |
| Approach: | They propose a confidence estimation approach for named entity recognition using noisy labels using local and global independence assumptions. |
| Outcome: | The proposed method marginalizes out labels of low confidence with a CRF model and integrates it into a self-training framework for boosting performance. |
Boosting LLM Agents with Recursive Contemplation for Effective Deception Handling (2024.findings-acl)
Copied to clipboard
Shenzhi Wang, Chang Liu, Zilong Zheng, Siyuan Qi, Shuo Chen, Qisen Yang, Andrew Zhao, Chaofei Wang, Shiji Song, Gao Huang
| Challenge: | Recent advances in large language models (LLMs) have led to significant success in using LLMs as agents. |
| Approach: | They propose a cognitive framework that incorporates first-order and second-order perspective transitions into LLMs to enhance their ability to identify and counteract deceptive information. |
| Outcome: | The proposed framework enhances LLMs’ ability to identify and counteract deceptive information without extra fine-tuning and data. |
Chain-of-Procedure: Hierarchical Visual-Language Reasoning for Procedural QA (2026.findings-acl)
Copied to clipboard
Guanhua Chen, Yutong Yao, Shenghe Sun, Ci-jun Gao, Shudong Liu, Lidia S. Chao, Feng Wan, Derek F. Wong
| Challenge: | Recent advances in vision-language models (VLMs) have achieved impressive results on standard image-text tasks, yet their capability in visual procedure question answering (VP-QA) remains largely unexplored. |
| Approach: | They propose a multimodal benchmark specifically designed for visual procedural reasoning that synergizes cross-modal procedure retrieval, context-aware step decomposition, and the next step prediction. |
| Outcome: | The proposed framework significantly outperforms baselines on visual procedure question answering (VP-QA) Experiments on six VLMs show that it performs better than baselines. |
VMSMO: Learning to Generate Multimodal Summary for Video-based News Articles (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that multimodal news can significantly improve users' sense of satisfaction for informativeness. |
| Approach: | They propose a task of Video-based Multimodal Summarization with Multimodal Output to solve this problem. |
| Outcome: | The proposed method can generate multimodal summaries with a single input . it can model the temporal dependency of video with semantic meaning of article . |
Taming Text-to-Image Synthesis for Novices: User-centric Prompt Generation via Multi-turn Guidance (2025.emnlp-main)
Copied to clipboard
Yilun Liu, Minggui He, Feiyu Yao, Yuhe Ji, Shimin Tao, Jingzhou Du, Justin Li, Jian Gao, Zhang Li, Hao Yang, Boxing Chen, Osamu Yoshie
| Challenge: | Existing solutions for text-to-image synthesis are sensitive on textual prompts, posing a challenge for novice users. |
| Approach: | They propose a dialogue-based TIS prompt generation model that emphasizes user experience for novice users. |
| Outcome: | The proposed model emphasizes user experience for novice users . it improves user-centricity score while maintaining a competitive quality of synthesized images. |
BrowseComp-Plus: A Fair and Disentangled Evaluation Benchmark for Deep Search Agents (2026.acl-long)
Copied to clipboard
Zijian Chen, Xueguang Ma, Shengyao Zhuang, Ping Nie, Kai Zou, Sahel Sharifymoghaddam, Andrew Liu, Joshua Green, Kshama Patel, Ruoxi Meng, Mingyi Su, Yanxi Li, Haoran Hong, Xinyu Shi, Xuye Liu, Hosna Oyarhoseini, Nandan Thakur, Crystina Zhang, Luyu Gao, Wenhu Chen, Jimmy Lin
| Challenge: | Existing benchmarks for deep search agents rely on blackbox web search APIs . dynamic and opaque web APIs hinder reproducibility and fair comparisons - authors . |
| Approach: | They propose a benchmark that employs a fixed corpus for controlled retrieval for deep search agents. |
| Outcome: | The new benchmark shows that agents that combine large language models with retrieval tools excel at complex, reasoning-intensive queries. |
Do Vision-Language Models Have Internal World Models? Towards an Atomic Evaluation (2025.findings-acl)
Copied to clipboard
Qiyue Gao, Xinyu Pi, Kevin Liu, Junrong Chen, Ruolan Yang, Xinqi Huang, Xinyu Fang, Lu Sun, Gautham Kishore, Bo Ai, Stone Tao, Mengyang Liu, Jiaxi Yang, Chao-Jung Lai, Chuanyang Jin, Jiannan Xiang, Benhao Huang, Zeming Chen, David Danks, Hao Su, Tianmin Shu, Ziqiao Ma, Lianhui Qin, Zhiting Hu
| Challenge: | Recent studies have evaluated and shown limitations in specific capabilities such as visual understanding, but a systematic evaluation of VLMs’ fundamental WM abilities remains absent. |
| Approach: | They propose a framework that assesses perception and prediction to provide an atomic evaluation of VLMs as WMs. |
| Outcome: | The proposed framework assesses perception and prediction abilities on 15 latest VLMs and compares them to human-level models. |
A Hybrid Neural Network Model for Commonsense Reasoning (D19-60)
Copied to clipboard
| Challenge: | a hybrid neural network (HNN) model for commonsense reasoning is proposed . it combines language models and semantic similarity models to achieve new state-of-the-art results . |
| Approach: | They propose a hybrid neural network model for commonsense reasoning . it combines a masked language model and a semantic similarity model . |
| Outcome: | The proposed model outperforms the WNLI, WSC and PDP60 benchmarks on three commonsense reasoning tasks. |
How to Write Summaries with Patterns? Learning towards Abstractive Summarization through Prototype Editing (D19-1)
Copied to clipboard
| Challenge: | Extensive experiments on a large-scale real-world text summarization dataset show that PESG achieves the state-of-the-art performance in terms of both automatic metrics and human evaluations. |
| Approach: | They propose a model that learns summary patterns and prototype facts from a prototype document . they use a fact checker to estimate mutual information between the input document and generated summary . |
| Outcome: | Experiments on a large-scale real-world text summarization dataset show that PESG achieves state-of-the-art performance. |
Slot-Gated Modeling for Joint Slot Filling and Intent Prediction (N18-2)
Copied to clipboard
| Challenge: | Existing approaches for slot filling and intent detection have independent attention weights, but they suffer from error propagation due to their independent models. |
| Approach: | They propose a slot gate that focuses on learning the relationship between intent and slot attention vectors to obtain better semantic frame results by the global optimization. |
| Outcome: | The proposed model significantly improves sentence-level semantic frame accuracy with 4.2% and 1.9% relative improvement compared to the attentional model on benchmark ATIS and Snips datasets respectively. |
Guiding AMR Parsing with Reverse Graph Linearization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) parsing aims to extract an abstract semantic graph from a sentence. |
| Approach: | They propose a new framework that allows for reversed linearization of AMR graphs . they propose to combine sequence-to-sequence approaches with a linearized graph . |
| Outcome: | The proposed framework outperforms the best AMR parser by 0.8 and 0.5 Smatch scores on the AMR 2.0 and AMR 3.0 datasets. |
ProMind-LLM: Proactive Mental Health Care via Causal Reasoning with Sensor Data (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for mental health risk assessment rely on subjective textual records . however, these uncertainties can cause inconsistent and unreliable predictions . |
| Approach: | They propose a method that integrates objective behavior data alongside subjective mental records for robust mental health risk assessment. |
| Outcome: | The proposed approach achieves significant improvements over general LLMs. |
CityNavAgent: Aerial Vision-and-Language Navigation with Hierarchical Semantic Planning and Global Memory (2025.acl-long)
Copied to clipboard
Weichen Zhang, Chen Gao, Shiquan Yu, Ruiying Peng, Baining Zhao, Qian Zhang, Jinqiang Cui, Xinlei Chen, Yong Li
| Challenge: | Existing ground VLN agents struggle in aerial VLLN due to the lack of predefined navigation graphs and the exponentially expanding action space in long-horizon exploration. |
| Approach: | They propose a large language model-empowered aerial VLN agent that decomposes the long-horizon task into sub-goals with different semantic levels. |
| Outcome: | The proposed method achieves state-of-the-art performance with significant improvement in continuous city environments. |
Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Understanding how jailbreaking works remains limited, hindering the development of effective defense strategies. |
| Approach: | They propose a new mechanism that adaptively constrains activations within the safety boundary and propose 'Activation Boundary Defense' to enhance its effectiveness. |
| Outcome: | The proposed defense achieves an average Defense Success Rate (DSR) of over 98% against various jailbreak attacks, with less than 2% impact on the model’s general capabilities. |
From Discrimination to Generation: Low-Resource Intent Detection with Language Model Instruction Tuning (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies fine-tune discriminative models on specific defined intent classes, preventing them from being directly adopted to new intent domains. |
| Approach: | They propose to use a pre-trained generative intent model to detect new intents from different domains with no parameter updates. |
| Outcome: | The proposed model outperforms baselines that need further fine-tuning or domain-specific samples. |
Infinite Babble: Inflating 3D Vision-Language Model Inference Overhead via Adversarial Geometric Perturbation (2026.findings-acl)
Copied to clipboard
| Challenge: | 3D Vision-Language Models (VLMs) are critical cognitive backbone for spatial intelligence, but their reliance on autoregressive decoding introduces a fundamental vulnerability regarding inference efficiency. |
| Approach: | They propose a framework that triggers computational and economic exhaustion in 3D-VLMs by injecting imperceptible noise that forces the model into a state of pathological verbosity. |
| Outcome: | The proposed framework amplifies output length and energy consumption by up to 6.45, demonstrating a potent capability to drain system resources. |
Code Reffix: A Benchmark for Reflection-Guided Code Repair with Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks focus on the repair generation capability of LLMs, lacking fine-grained evaluation of reflection. |
| Approach: | They propose a benchmark with oracle reflections and a dual-task protocol to decouple evaluation of reflection from repair. |
| Outcome: | The proposed benchmarks show that underperforming reflection capabilities remain a bottleneck for code repair. |
FashionKLIP: Enhancing E-Commerce Image-Text Retrieval with Fashion Multi-Modal Conceptual Knowledge Graph (2023.acl-industry)
Copied to clipboard
Xiaodan Wang, Chengyu Wang, Lei Li, Zhixu Li, Ben Chen, Linbo Jin, Jun Huang, Yanghua Xiao, Ming Gao
| Challenge: | Recent advances in visual-language pre-trained (VLP) models have greatly improved cross-modal retrieval performance . however, the fine-grained interactions between objects from different modalities are far from well-established . e-commerce domain lacks sufficient training data and fine-granular cross-modulal knowledge . |
| Approach: | They propose a visual-language pre-trained (VLP) image-text retrieval model that integrates cross-modal knowledge into the model to improve performance. |
| Outcome: | The proposed model improves performance on e-commerce image-text retrieval task by a large margin. |
Scaling Beyond Context: A Survey of Multimodal Retrieval-Augmented Generation for Document Understanding (2026.acl-long)
Copied to clipboard
Sensen Gao, Shanshan Zhao, Xu Jiang, Lunhao Duan, Yong Xien Chng, Qing-Guo Chen, Weihua Luo, Kaifu Zhang, Jia-Wang Bian, Mingming Gong
| Challenge: | Document understanding is critical for applications from financial analysis to scientific discovery. |
| Approach: | They propose a taxonomy based on domain, retrieval modality, and granularity and review advances involving graph structures and agentic frameworks. |
| Outcome: | The proposed model enables holistic retrieval and reasoning across all modalities, unlocking comprehensive document intelligence. |
ProCeedRL: Process Critic with Explorative Demonstration Reinforcement Learning for LLM Agentic Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit exceptional reasoning capabilities, driven by Reinforcement Learning with Verifiable Rewards (RLVR). |
| Approach: | They propose a method that uses a process-level critic to monitor interactions in real time, incorporating reflection-based demonstrations to guide agents in stopping the accumulation of errors. |
| Outcome: | The proposed approach exceeds the model’s saturated exploration performance and achieves superior performance on complex deep search and embodied tasks. |
Structure-aware Fine-tuning for Code Pre-trained Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CodePTMs are mainly structure-free and structurebased, but how to fine-tune them remains a challenge. |
| Approach: | They propose a plug-and-play fine-tuning method that incorporates structural knowledge into pre-trained code models. |
| Outcome: | The proposed method can benefit CodePTMs more with limited training data. |
Multi-Task Deep Neural Networks for Natural Language Understanding (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to learning vector-space representations of text are multitask learning and language model pre-training. |
| Approach: | They propose a multi-task deep neural network (MT-DNN) that leverages cross-task data and incorporates a pre-trained bidirectional transformer language model. |
| Outcome: | The proposed model achieves state-of-the-art on ten NLU tasks and pushes the GLUE benchmark to 82.7% (2.2% absolute improvement) |
GUITester: Enabling GUI Agents for Exploratory Defect Discovery (2026.findings-acl)
Copied to clipboard
| Challenge: | Exploratory GUI testing is essential for software quality but suffers from high manual costs. |
| Approach: | They propose a framework that decouples navigation from verification via two modules . they propose 143 tasks and a GUITestBench benchmark that features 26 defects . |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks in 143 tasks and 26 defects. |
MPCoder: Multi-user Personalized Code Generator with Explicit and Implicit Style Representation Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing Large Language Models are usually generalized with large programming corpus, therefore the generated code is difficult to adapt to personalized and/or customized requests. |
| Approach: | They propose a method to use Large Language Models to generate personalized code for multiple users. |
| Outcome: | The proposed model can generate personalized code for multiple users . it can be used to improve code generation and reduce maintenance costs. |
Is Your LLM Outdated? A Deep Look at Temporal Generalization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate large language models are limited due to their inherent dynamic nature and the inherent dynamicity of language and information. |
| Approach: | They introduce a new evaluation framework that employs fresh text and event prediction for assessing LLMs’ temporal adaptability. |
| Outcome: | The proposed framework shows significant temporal biases and a decline in performance over time. |
Simple and Effective Text Matching with Richer Alignment Features (P19-1)
Copied to clipboard
| Challenge: | Existing models only use a single inter-sequence alignment layer to make full use of this process. |
| Approach: | They propose to keep three key features available for inter-sequence alignment . they conduct experiments on four well-studied benchmark datasets . |
| Outcome: | The proposed model is able to perform on four well-studied datasets with fewer parameters and the inference speed is at least 6 times faster than similar models. |
Word Form Matters: LLMs’ Semantic Reconstruction under Typoglycemia (2025.findings-acl)
Copied to clipboard
| Challenge: | Typoglycemia is a phenomenon where people can read words even when the middle letters of the words are scrambled. |
| Approach: | They propose a reliable metric to quantify the degree of semantic reconstruction and validate its effectiveness. |
| Outcome: | The proposed metric quantifies the degree of semantic reconstruction and validates its effectiveness. |
Decoding Scientific Experimental Images: The SPUR Benchmark for Perception, Understanding, and Reasoning (2026.acl-long)
Copied to clipboard
Junpeng Ding, Zichen Tang, Haihong E, Mengyuan Ji, Yang Liu, Haolin Tian, Haiyang Sun, Pengqi Sun, Yang Xu, Yichen Liu, Haocheng Gao, Zijie Xi, Ruomeng Jiang, Peizhi Zhao, Rongjin Li, Yuanze Li, Jiacheng Liu, Zhongjun Yang, Jintong Chen, Siying Lin
| Challenge: | Xu and Peng, 2025) . . SPUR is a comprehensive benchmark for scientific experimental image perception, understanding, and reasoning, comprising 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images. |
| Approach: | They propose to use 4,264 question-answering (QA) pairs derived from 1,084 expert-curated images to evaluate the visual perception of multimodal large language models (MLLMs) . they also propose to utilize cross-panel relation understanding to evaluate MLLM’s ability to decipher intricate cross-panel relations. |
| Outcome: | The proposed model is based on 4,264 question-answering pairs derived from 1,084 expert-curated images. |
Towards a Unified Framework for Reference Retrieval and Related Work Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for related work generation use human-annotated references as information sources. |
| Approach: | They propose a model which combines reference retrieval and related work generation processes in a unified framework based on the large language model. |
| Outcome: | The proposed model outperforms the state-of-the-art models on two wide-applied datasets. |
ZigZagKV: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to accelerate inference of Large Language models (LLMs) are limited in their ability to retain key tokens as input length increases. |
| Approach: | They propose a method that leverages layer uncertainty to allocate budget size for each layer to reduce memory usage. |
| Outcome: | The proposed method reduces memory usage of the KV caches to only 20% when compared to full KV inference while achieving nearly lossless performance. |
AutoSDT: Scaling Data-Driven Discovery Tasks Toward Open Co-Scientists (2025.emnlp-main)
Copied to clipboard
Yifei Li, Hanane Nour Moussa, Ziru Chen, Shijie Chen, Botao Yu, Mingyi Xue, Benjamin Burns, Tzu-Yao Chiu, Vishal Dey, Zitong Lu, Chen Wei, Qianheng Zhang, Tianyu Zhang, Song Gao, Xuhui Huang, Xia Ning, Nesreen K. Ahmed, Ali Payani, Huan Sun
| Challenge: | AutoSDT-5K is the only automatically collected and the largest open dataset for data-driven scientific discovery. |
| Approach: | They propose an automatic pipeline that collects high-quality coding tasks in real-world data-driven discovery workflows. |
| Outcome: | The proposed pipeline synthesizes accurate tasks and tasks from a dataset of 5,404 tasks covering four scientific disciplines and 756 Python packages. |
Understanding LLMs’ Cross-Lingual Context Retrieval: How Good It Is And Where It Comes From (2025.emnlp-main)
Copied to clipboard
| Challenge: | Cross-lingual context retrieval is a fundamental aspect of cross-lingual alignment, but the performance and mechanism of it for large language models (LLMs) remains unclear. |
| Approach: | They evaluate cross-lingual context retrieval of over 40 large language models . they use cross-linguistic machine reading comprehension as a representative scenario . |
| Outcome: | The results show that open LLMs show strong cross-lingual context retrieval ability . the results also show that their oracle performances improve after training . |
Budgeted Policy Learning for Task-Oriented Dialogue Systems (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to integrating reinforcement learning into task-oriented dialogue systems require a fixed, small amount of user interactions to learn. |
| Approach: | They propose a budget-conscious scheduling approach that optimizes a fixed, small amount of user interactions for dialogue agent learning. |
| Outcome: | The proposed approach improves on a movie-ticket booking task with simulated and real users. |
Pass-Tuning: Towards Structure-Aware Parameter-Efficient Tuning for Code Representation Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Code pre-trained models have been proposed and widely applied in the domain of code intelligence. |
| Approach: | They propose a method that uses a plug-and-play graph neural network module as a tunable prefix to exploit structural information of source code. |
| Outcome: | The proposed method exploits structural information of source code and could replace full fine-tuning. |
Moderation Matters: Measuring Conversational Moderation Impact in English as a Second Language Group Discussion (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing tools for ESL assessment focus on writing skills and lack in support for dynamic spoken interactions. |
| Approach: | They propose an approach that integrates automatic ESL dialogue assessment and a framework that categorizes moderation strategies to assess conversational engagement and moderation effectiveness. |
| Outcome: | The proposed approach integrates automatic ESL dialogue assessment and categorizes moderation strategies. |
Enabling Large Language Models to Generate Text with Citations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing work relies on commercial search engines and human evaluation, making it difficult to reproduce and compare different modeling approaches. |
| Approach: | They propose a new generation paradigm that requires large language models to provide citations to one or a few text passages for any statement they generate. |
| Outcome: | The proposed model improves factual correctness and verifiability of large language models by providing citations to a set of questions and retrieval corpora and generating answers with citation. |
MIND: A Large-scale Dataset for News Recommendation (2020.acl-main)
Copied to clipboard
Fangzhao Wu, Ying Qiao, Jiun-Hung Chen, Chuhan Wu, Tao Qi, Jianxun Lian, Danyang Liu, Xing Xie, Jianfeng Gao, Winnie Wu, Ming Zhou
| Challenge: | Personalized news recommendation is an important technique for personalized news service. |
| Approach: | They propose to build a large-scale news recommendation dataset from Microsoft News . they demonstrate that news recommendation relies on the quality of news content understanding . |
| Outcome: | The proposed dataset contains 1 million users and more than 160k English news articles, each of which has rich textual content such as title, abstract and body. |
Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time (2026.findings-acl)
Copied to clipboard
Mingkuan Zhao, Yide Gao, Wentao Hu, Suquan Chen, Tianchen Huang, Zhenhua An, Zetao Chang, Xiayu Sun, Yuheng Min
| Challenge: | Existing mitigation strategies rely on suppressing specific neuron activations or employing computationally expensive contrastive decoding mechanisms, which often result in increased perplexity or significantly elevated inference latency. |
| Approach: | They propose a lightweight inference-time intervention method grounded in the perspective of residual stream signal dynamics to resolve the signal attenuation of external evidence during its propagation through deep networks. |
| Outcome: | The proposed method improves contextual faithfulness across multiple factual consistency and strong knowledge-conflict tasks while maintaining the model’s general language understanding capabilities. |
CAT-probing: A Metric-based Approach to Interpret How Pre-trained Models for Programming Language Attend Code Structure (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing code pre-trained models fail to consider inherent characteristics of codes . Existing methods to interpret code pretrained model fail to take into account inherent characteristics . |
| Approach: | They propose a probing method to quantitatively interpret how CodePTMs attend code structure. |
| Outcome: | The proposed method denoises input code sequences and measures commonality between token-level attention scores and pair-wise distances between corresponding AST nodes. |
Apertus: Democratizing Open and Compliant LLMs for Global Language Environments (2026.acl-long)
Copied to clipboard
Alejandro Hernández-Cano, Alexander Hägele, Allen Hao Huang, Angelika Romanou, Antoni-Joan Solergibert, Barna Pásztor, Bettina Messmer, Dhia Garbaya, Eduard Frank Ďurech, Ido Hakimi, Juan Garcia Giraldo, Mete Ismayilzada, Negar Foroutan, Skander Moalla, Tiancheng Chen, Vinko Sabolčec, Yixuan Xu, Michael Aerni, Badr AlKhamissi, Inés Altemir Marinas, Mohammad Hossein Amani, Matin Ansaripour, Ilia Badanin, Harold Benoit, Emanuela Boros, Nicholas John Browning, Fabian Bösch, Maximilian Böther, Niklas Canova, Camille Challier, Clément Charmillot, Jonathan Coles, Jan Milan Deriu, Arnout Devos, Lukas Drescher, Daniil Dzenhaliou, Maud Ehrmann, Dongyang Fan, Simin Fan, Silin Gao, Miguel Gila, María Grandury, Diba Hashemi, Alexander Miserlis Hoyle, Jiaming Jiang, Mark Klein, Andrei Kucharavy, Anastasiia Kucherenko, Frederike Lübeck, Roman Machacek, Theofilos Ioannis Manitaras, Andreas Marfurt, Kyle Matoba, Simon Matrenok, Henrique Mendonça, Fawzi Roberto Mohamed, Syrielle Montariol, Luca Mouchel, Sven Najem-Meyer, Jingwei Ni, Gennaro Oliva, Matteo Pagliardini, Elia Palme, Andrei Panferov, Léo Paoletti, Marco Passerini, Ivan Pavlov, Auguste Poiroux, Kaustubh Ponkshe, Nathan Ranchin, Javier Rando, Mathieu Sauser, Jakhongir Saydaliev, Mukhammadali Sayfiddinov, Marian Schneider, Stefano Schuppli, Marco Scialanga, Andrei Semenov, Kumar Shridhar, Raghav Singhal, Anna Sotnikova, Alexander Sternfeld, Ayush Kumar Tarun, Paul Teiletche, Jannis Vamvas, Xiaozhe Yao, Hao Zhao, Alexander Ilic, Ana Klimovic, Andreas Krause, Caglar Gulcehre, David Rosenthal, Elliott Ash, Florian Tramèr, Joost VandeVondele, Livio Veraldi, Martin Rajman, Thomas C. Schulthess, Torsten Hoefler, Antoine Bosselut, Martin Jaggi, Imanol Schlag
| Challenge: | Apertus is a fully open suite of large language models (LLMs) designed to address responsibility shortcomings in today’s open model ecosystem, namely data responsibility and global representation. |
| Approach: | They propose to release a fully open suite of large language models (LLMs) that address data responsibility and global representation shortcomings in today’s open model ecosystem. |
| Outcome: | The proposed model is pretrained on openly available data and suppresses verbatim recall of data while retaining task performance. |
Human-Agent Collaborative Paper-to-Page Crafting (2026.findings-acl)
Copied to clipboard
Qianli Ma, Siyu Wang, Chen Yilin, Yinhao Tang, Yixiang Yang, Chang Guo, Bingjie Gao, Zhening Xing, Yanan Sun, Zhipeng Zhang
| Challenge: | Existing approaches to create project pages from academic papers have focused on static slides and posters, but the dynamic nature of webpages remains an unaddressed challenge. |
| Approach: | They propose a novel multi-agent system that deconstructs paper-to-page creation into a coarse-to fine pipeline from narrative planning to multimodal content generation and interactive rendering. |
| Outcome: | The proposed system generates high-quality, visually appealing pages in under 15 minutes for less than $0.1 . |
PSP: Pre-trained Soft Prompts for Few-Shot Abstractive Summarization (2022.coling-1)
Copied to clipboard
| Challenge: | Experimental results show that our method outperforms full-model tuning in few-shot abstractive summarization tasks. |
| Approach: | They propose a soft prompts architecture with prompt pre-training and prompt fine-tuning paradigm to support few-shot abstractive summarization. |
| Outcome: | The proposed model outperforms Prompt Tuning and Profix-Tuning on CNN/DailyMail and XSum datasets and outperfies Profix Tuning by a large margin. |
NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning (2021.starsem-1)
Copied to clipboard
| Challenge: | Currently, symbolic and deep learning approaches to NLI are receiving less attention. |
| Approach: | They propose a symbolic-based inference framework that integrates symbolic reasoning and semantic formalism to solve NLI tasks. |
| Outcome: | The proposed framework improves accuracy on the NLI task and on the SICK and MED datasets. |
IAD: In-Context Learning Ability Decoupler of Large Language Models in Meta-Training (2024.lrec-main)
Copied to clipboard
| Challenge: | In-context Learning (ICL) is a paradigm in which LLMs acquire task-specific knowledge by processing input-output pairs provided as prompts. |
| Approach: | They propose an In-context learning Ability Decoupler to separate ICL ability from general ability of LLMs in meta-training phase . they first identify parameters suitable for ICL by transference-driven gradient importance and propose a new max-margin loss to emphasize the separation of the two abilities. |
| Outcome: | The proposed model separates the ICL ability from the general ability of LLMs in the meta-training phase, where the I-related parameters are tuned to adapt for ICL tasks. |
Self-Adjust Softmax (2025.emnlp-main)
Copied to clipboard
Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi, Jing Xiong, Xiaozhe Ren, Chao Huang, Zhenguo Li, Yu Li
| Challenge: | Usually, tokens with larger attention scores are important for the final prediction. |
| Approach: | They propose to modify softmax(z) to z softmax and its normalized variant to improve the Transformer attention mechanism by making minor adjustments to the softmax function. |
| Outcome: | The proposed model provides enhanced gradient properties compared to the vanilla softmax function. |
Defining and Evaluating Visual Language Models’ Basic Spatial Abilities: A Perspective from Psychometrics (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies assessing the spatial abilities of VLMs lack a solid theoretical foundation and lack measurable data. |
| Approach: | They propose a psychometric framework defining five basic spatial abilities in Visual Language Models. |
| Outcome: | The proposed framework defines five basic spatial abilities in Visual Language Models (VLMs) it provides a comprehensive evaluation benchmark and methodological perspective for embodied AI development . |
Clear Up Confusion: Iterative Differential Generation for Fine-grained Intent Detection with Contrastive Feedback (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies on fine-grained intent detection have focused on collecting large-scale and high-quality samples via crowdsourcing resulting in data scarcity. |
| Approach: | They propose an iterative differential generation framework with contrastive feedback to generate high-quality pseudo samples and accurately capture the crucial nuances in target class distribution. |
| Outcome: | The proposed framework generates high-quality pseudo samples and captures crucial nuances in target class distribution. |
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks (2024.findings-emnlp)
Copied to clipboard
Jack Gallifant, Shan Chen, Pedro Moreira, Nikolaj Munch, Mingye Gao, Jackson Pond, Leo Anthony Celi, Hugo Aerts, Thomas Hartvigsen, Danielle Bitterman
| Challenge: | Medical knowledge is context-dependent and requires consistent reasoning across various natural language expressions of semantically equivalent phrases. |
| Approach: | They create a robustness dataset to evaluate performance differences on medical benchmarks . they swap brand and generic drug names using physician expert annotations based on medical terminology . |
| Outcome: | The proposed model shows a consistent performance drop of 1-10% on medical benchmarks. |
LeTS: Learning to Think-and-Search via Process-and-Outcome Reward Hybridization (2025.emnlp-main)
Copied to clipboard
Qi Zhang, Shouqing Yang, Lirong Gao, Hao Chen, Xiaomeng Hu, Jinglei Chen, Jiexiang Wang, Sheng Guo, Bo Zheng, Haobo Wang, Junbo Zhao
| Challenge: | Recent research focuses on integrating reasoning capabilities into the realm of retrieval-augmented generation (RAG) via outcome-supervised reinforcement learning (RL). |
| Approach: | They propose a process-level reward module to mitigate the unawareness of intermediate reasoning steps in outcome-level supervision without additional annotation. |
| Outcome: | The proposed framework can boost LLMs’ reasoning ability by integrating external knowledge sources through retrieval-augmented generation (RAG) The proposed model can mitigate the unawareness of intermediate reasoning steps in outcome-level supervision without additional annotation. |
LitSearch: A Retrieval Benchmark for Scientific Literature Search (2024.emnlp-main)
Copied to clipboard
| Challenge: | Literature search questions pose significant challenges for modern retrieval systems . a lack of domain expertise and reasoning through lengthy papers is a challenge . |
| Approach: | They propose a retrieval benchmark for literature search queries using inline citations from papers and questions about recently published papers. |
| Outcome: | The proposed retrieval benchmarks outperform state-of-the-art retrieval models and reranking pipelines. |
Mixture of insighTful Experts (MoTE): The Synergy of Reasoning Chains and Expert Mixtures in Self-Alignment (2025.acl-long)
Copied to clipboard
Zhili Liu, Yunhao Gou, Kai Chen, Lanqing Hong, Jiahui Gao, Fei Mi, Yu Zhang, Zhenguo Li, Xin Jiang, Qun Liu, James Kwok
| Challenge: | Recent studies show that reasoning abilities contribute significantly to model safety, while integrating Mixture-of-Experts (MoE) architectures can further enhance alignment. |
| Approach: | They propose a framework that synergistically combines reasoning chains and expert mixtures to improve self-alignment. |
| Outcome: | The proposed framework improves model safety, jailbreak resistance, and over-refusal capabilities, achieving performance comparable to OpenAI’s state-of-the-art o1 model. |
AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have focused on factual correctness, semantic grounding, visual reasoning, or multimodal large language models. |
| Approach: | They propose a benchmark to assess AICA, which integrates perception, reasoning, and generation into a unified framework. |
| Outcome: | The proposed framework corrects intensity errors and significantly enhances descriptive depth. |
CoD, Towards an Interpretable Medical Agent using Chain of Diagnosis (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) are proving to be effective in medical automatic diagnosis, but their interpretability remains unaddressed. |
| Approach: | They propose to use a "Chain-of-Diagnosis" approach to enhance the interpretability of medical automatic diagnosis by outputting the disease confidence distribution. |
| Outcome: | The proposed model outperforms other LLMs on automatic diagnostic tasks across three real-world benchmarks and provides interpretability while ensuring controllability in diagnostic rigor. |
Analyzing and Modeling LLM Response Lengths with Extreme Value Theory: Anchoring Effects and Hybrid Distributions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches treat length as an incidental output property rather than a statistically regular phenomenon worthy of rigorous modeling. |
| Approach: | They propose a statistical framework for modeling and controlling large language model response lengths using extreme value theory and cross-validation on Qwen and DeepSeek architectures. |
| Outcome: | The proposed model improves tail fit and generalizability while maintaining generalizzability. |
JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing text-to-text methods struggle with issues such as generalization, robustness, and controllability due to their lack of explicit task structures. |
| Approach: | They propose a structure-to-structure approach that uses JSON structures to represent tasks. |
| Outcome: | The proposed method outperforms TextTuning in terms of performance, robustness, and controllability across different scenarios. |
Learn to Relax with Large Language Models: Solving Constraint Optimization Problems via Bidirectional Coevolution (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Model (LLM)-based optimization has shown promise for autonomous problem solving, but most approaches cast LLMs as passive constraint checkers rather than proactive strategy designers. |
| Approach: | They propose an end-to-end Automated Constraint Optimization method that tightly couples operations-research principles of constraint relaxation with LLM reasoning. |
| Outcome: | Extensive experiments on three challenging COP benchmarks validate AutoCO’s consistent effectiveness and superior performance, especially in hard regimes where current methods degrade. |
VLMInferSlow: Evaluating the Efficiency Robustness of Large Vision-Language Models as a Service (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies evaluate efficiency robustness of vision-language models under unrealistic assumptions, requiring access to model architecture and parameters. |
| Approach: | They propose a novel approach to evaluate VLM efficiency robustness in a realistic black-box setting. |
| Outcome: | The proposed approach generates adversarial images with imperceptible perturbations, increasing the computational cost by up to 128.47%. |
Continual Few-shot Intent Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing intent detection systems are trained with lots of labeled data over a predefined set of intent classes. |
| Approach: | They propose a prefix-guided lightweight encoder with three auxiliary strategies to prevent catastrophic forgetting and negative knowledge transfer across tasks. |
| Outcome: | The proposed system prevents catastrophic forgetting and encourages positive knowledge transfer across tasks. |
Harnessing the Power of Large Language Model for Uncertainty Aware Graph Processing (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for graph processing rely on assumptions about data relations that are inadequate when handling large and complex graph data. |
| Approach: | They propose a large language model enhanced by an uncertainty-aware module to provide a confidence score on the generated graph data. |
| Outcome: | The proposed approach surpasses state-of-the-art algorithms by a substantial margin on ten datasets. |
Conjoin after Decompose: Improving Few-Shot Performance of Named Entity Recognition (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing prompt-based NER models fail to detect entity boundaries, causing performance degradation. |
| Approach: | They propose a model which consists of a BART encoder and a parabiotic decoder and propose ' boundary expansion strategy' to enhance the model's capability in entity type classification. |
| Outcome: | The proposed model can achieve significant performance gains over state-of-the-art models. |
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding (2026.findings-acl)
Copied to clipboard
Yuqi Yang, Weiqi Wang, Baixuan Xu, Wei Fan, Qing Zong, Chunkit Chan, Zheye Deng, Xin Liu, Yifan Gao, Changlong Yu, Chen Luo, Yang Li, Zheng Li, Qingyu Yin, Bing Yin, Yangqiu Song
| Challenge: | Existing models fail to capture and model customer intention effectively because of insufficient information exploitation and only apparent information like descriptions and titles are used. |
| Approach: | They propose to exploit existing session data to capture and model intention in E-commerce product purchase sessions using a multimodal benchmark. |
| Outcome: | The proposed framework can bridge the gap between intention understanding in simplified research cases like co-buy intention and more complex yet practical scenarios like session history. |
Pre-training CLIP against Data Poisoning with Optimal Transport-based Matching and Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that Contrastive Language-Image Pre-training (CLIP) models are vulnerable to data poisoning and backdoor attacks due to massive training image-caption pairs crawled from the Internet. |
| Approach: | They propose an Optimal Transport-based framework to reconstruct image-caption pairs and propose an optimal transport-based distance measure to re-assign new captions based on the proposed optimal transport distance. |
| Outcome: | The proposed framework reduces the attack success rates of poisoning attacks to 0% in most cases. |
Dr3: Ask Large Language Models Not to Give Off-Topic Answers in Open Domain Multi-Hop Question Answering (2024.lrec-main)
Copied to clipboard
Yuan Gao, Yiheng Zhu, Yuanbin Cao, Yinzhi Zhou, Zhen Wu, Yujie Chen, Shenglan Wu, Haoyuan Hu, Xinyu Dai
| Challenge: | Open Domain Multi-Hop Question Answering (ODMHQA) is one of the most challenging tasks in Natural Language Processing (NLP) |
| Approach: | They propose a mechanism that leverages the intrinsic capabilities of Large Language Models to judge whether the generated answers are off-topic. |
| Outcome: | The proposed method reduces the occurrence of off-topic answers by nearly 13%, improving the performance in Exact Match (EM) by nearly 3% compared to the baseline method without the Dr3 mechanism. |
Automatic Term Name Generation for Gene Ontology: Task and Dataset (2020.findings-emnlp)
Copied to clipboard
Yanjian Zhang, Qin Chen, Yiteng Zhang, Zhongyu Wei, Yixu Gao, Jiajie Peng, Zengfeng Huang, Weijian Sun, Xuanjing Huang
| Challenge: | Gene Ontology (GO) terms are used to describe gene function in biology and bio-medicine. |
| Approach: | They propose a task to generate term names for GO and build a large-scale benchmark dataset. |
| Outcome: | The proposed model outperforms baselines by incorporating the relations between genes, words and terms for term name generation. |
ContrastKV: Robust KV Cache Eviction via Contrastive Signal Fusion for Multi-Query Generalization (2026.acl-long)
Copied to clipboard
| Challenge: | Existing query-agnostic approaches rely on a single proxy query, leading to fragile eviction decisions under high evict ratios. |
| Approach: | They propose a query-agnostic KV cache eviction algorithm that exploits complementary semantic and non-semantic signals. |
| Outcome: | Experiments show that the proposed algorithm outperforms state-of-the-art methods while retaining up to 92% accuracy with only 20% of the KV cache budget. |
When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario (2023.findings-acl)
Copied to clipboard
| Challenge: | Large pre-trained language models (PLMs) are expensive and may not be open-sourced due to commercial considerations and potential risks of misuse. |
| Approach: | They propose to introduce gradient descent into black-box tuning scenario . they propose a method which integrates gradient descent and derivative-free optimization . |
| Outcome: | The proposed method achieves significant performance gains over previous state-of-the-art methods. |
Polymorphic Universal Transformer (2026.acl-long)
Copied to clipboard
Yilong Chen, Zitian Gao, Yihao Xiao, Jason Klein Liu, Xinyu Yang, Yifan Luo, Haoming Luo, Zhengmao Ye, Tingwen Liu, Ran Tao, Bryan Dai
| Challenge: | Compute Distribution Skew is a pathological phenomenon in ultra-deep recurrent models . it causes over-smoothing, representation rank collapse, and degraded reasoning performance. |
| Approach: | They propose a dynamic architecture that redefines recursive computation by decoupling parameter count from depth. |
| Outcome: | The proposed model significantly improves representation rank and reasoning robustness while reducing computation by 64.7%. |
VoiceBench: Benchmarking LLM-Based Voice Assistants (2026.tacl-1)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have enabled real-time speech interactions through LLMs. |
| Approach: | They propose a benchmark specifically designed to assess LLM-based voice assistants. |
| Outcome: | The proposed benchmark measures the performance of LLM-based voice assistants across eight tasks. |
EfficientQAT: Efficient Quantization-Aware Training for Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Quantization-aware training (QAT) is a low-bit training solution that requires substantial training resources. |
| Approach: | They propose an algorithm that reduces memory consumption by low-bit representations with minimal accuracy loss. |
| Outcome: | EfficientQAT achieves 2-bit Llama-2-70B model on single GPU in 41 hours . compared to previous methods, it obtains model with less than 3 points accuracy degradation . |
Language-specific Effects on Automatic Speech Recognition Errors for World Englishes (2022.coling-1)
Copied to clipboard
| Challenge: | Existing systems are not able to meet the needs of speakers of different demographic groups. |
| Approach: | They propose to analyze the performance of Otter’s automatic captioning system on native and non-native English speakers of different language background through a linguistic analysis of segment-level errors. |
| Outcome: | The proposed system predicts certain errors from the phonological structure of a speaker’s native language. |
VenusFactory: An Integrated System for Protein Engineering with Data Retrieval and Language Model Fine-Tuning (2025.acl-demo)
Copied to clipboard
Yang Tan, Chen Liu, Jingyuan Gao, Wu Banghao, Mingchen Li, Ruilin Wang, Lingrong Zhang, Huiqun Yu, Guisheng Fan, Liang Hong, Bingxin Zhou
| Challenge: | Pre-trained protein language models have been used in protein engineering, but their adoption is limited due to data collection, task benchmarking, and application challenges. |
| Approach: | They propose a versatile engine that integrates biological data retrieval, standardized task benchmarking, and modular fine-tuning of PLMs. |
| Outcome: | The proposed engine integrates biological data retrieval, task benchmarking, and modular fine-tuning of PLMs. |
GazeVQA: A Video Question Answering Dataset for Multiview Eye-Gaze Task-Oriented Collaborations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the use of exocentric and egocentric videos in video question answering are focusing on eye-gaze information. |
| Approach: | They propose a task-oriented VQA dataset that captures eye-gaze information . they propose assisting models that ground the perceptual input into semantic information based on three different answer types . |
| Outcome: | The proposed model can ground the perceptual input into semantic information while reducing ambiguities. |
Beyond Profile: From Surface-Level Facts to Deep Persona Simulation in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to persona simulation large language models (LLMs) focus on learning basic biographical information, or using limited role-play dialogue datasets to capture a character’s responses. |
| Approach: | They propose to train characters using a linguistic structure and a style-tuning mechanism that allows a general linguistic style expert to collaborate with other task-specific experts to better understand their thoughts. |
| Outcome: | The proposed model outperforms baselines on linguistic accuracy and opinion comprehension on three tasks for Lu Xun's essay collection. |
SimCSE: Simple Contrastive Learning of Sentence Embeddings (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for learning universal sentence embeddings are based on unsupervised approaches with only dropout as noise. |
| Approach: | They propose an unsupervised approach that takes an input sentence and predicts itself in a contrastive objective with only standard dropout used as noise. |
| Outcome: | The proposed framework performs on par with previous supervised approaches and can produce superior sentence embeddings from unlabeled or labeled data. |
Towards Generating Long and Coherent Text with Multi-Level Latent Variable Models (P19-1)
Copied to clipboard
| Challenge: | Variational autoencoders (VAEs) have received much attention as an end-to-end architecture for text generation with latent variables. |
| Approach: | They propose to leverage several multi-level structures to learn a variational autoencoder model for generating long, and coherent text. |
| Outcome: | The proposed model produces more coherent and less repetitive long text compared to baselines and mitigates posterior collapse issue. |
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models fail to generalize to unseen data. |
| Approach: | They propose a framework for robust and efficient fine-tuning for pre-trained models . proposed framework achieves new state-of-the-art performance on a number of NLP tasks . |
| Outcome: | The proposed framework outperforms the state-of-the-art T5 model on GLUE, SNLI, SciTail and ANLI. |
Long-Context Language Modeling with Parallel Context Encoding (2024.acl-long)
Copied to clipboard
| Challenge: | Existing long-context models degenerate with retrieved contexts. |
| Approach: | They propose a framework that can be applied to existing decoder-only LLMs for context expansion. |
| Outcome: | The proposed framework can be applied to any existing decoder-only LLMs for context expansion. |
Parameter-Efficient Fine-Tuning via Circular Convolution (2025.findings-acl)
Copied to clipboard
| Challenge: | Low-Rank Adaptation (LoRA) has gained popularity for fine-tuning large foundation models, but its intrinsic low-rank characteristic may limit its performance. |
| Approach: | They propose a low-rank adaptive method that uses low-ranked matrices to represent weight changes. |
| Outcome: | The proposed method reduces trainable parameters and mitigates heavy memory consumption associated with full delta matrices by sequentially multiplying mathbf A and mathbb B with the activation. |
Reader-Guided Passage Reranking for Open-Domain Question Answering (2021.findings-acl)
Copied to clipboard
| Challenge: | Current open-domain question answering systems follow a Retriever-Reader architecture . current systems do not use a reranker, which reranked passages based on top predictions of the reader . |
| Approach: | They propose a reader-guIDEd reranking method that reranked passages based on top predictions . they show that RIDER achieves 10 to 20 absolute gains in top-1 retrieval accuracy . |
| Outcome: | The proposed method achieves 10 to 20 gains in top-1 retrieval accuracy and 1 to 4 Exact Match gains without training. |