Papers by Yin Li
Graph Reasoning for Question Answering with Triplet Retrieval (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to answer complex questions require reasoning over knowledge graphs (KGs) state-of-the-art methods constrain retrieved knowledge in local subgraphs and discard more diverse triplets that are disconnected but useful for question answering. |
| Approach: | They propose a method to retrieve the most relevant triplets from KGs and then rerank them, which are then concatenated with questions to be fed into language models. |
| Outcome: | The proposed method outperforms state-of-the-art methods on commonsenseQA and OpenbookQA datasets with 4.6% absolute accuracy. |
LLatrieval: LLM-Verified Retrieval for Verifiable Generation (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models struggle with factual errors and often produce non-factual and fabricated content. |
| Approach: | They propose to use large language models to generate text with supporting documents to enable the user to flexibly verify the answer. |
| Outcome: | Experiments on ALCE show that LLatrieval significantly outperforms extensive baselines and achieves state-of-the-art results. |
Too Long, Do Re-weighting for Efficient LLM Reasoning Compression (2026.acl-long)
Copied to clipboard
Zhong-Zhi Li, Xiao Liang, Zihao Tang, Lei Ji, Peijie Wang, Haotian Xu, Xing W, Haizhen Huang, Weiwei Deng, Yeyun Gong, Zhijiang Guo, Xiao Liu, Fei Yin, Cheng-Lin Liu
| Challenge: | Large Language Models (LLMs) have recently achieved remarkable progress on complex reasoning tasks by leveraging extended Chain-of-Thought (CoT) techniques. |
| Approach: | They propose a method that uses Extended Chain-of-Thought (EFT) to reduce the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
| Outcome: | The proposed method reduces the number of output tokens by nearly 40% while maintaining the accuracy of the reasoning. |
Gödel Agent: A Self-Referential Agent Framework for Recursively Self-Improvement (2025.acl-long)
Copied to clipboard
| Challenge: | Existing agentic systems cannot search the whole design space due to the restriction of human-designed components. |
| Approach: | They propose a Gödel Agent framework that allows agents to recursively improve themselves without relying on fixed algorithms or fixed algorithms. |
| Outcome: | The proposed framework surpasses manual crafted agents in performance, efficiency, and generalizability. |
LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing (2024.emnlp-main)
Copied to clipboard
Jiangshu Du, Yibo Wang, Wenting Zhao, Zhongfen Deng, Shuaiqi Liu, Renze Lou, Henry Zou, Pranav Narayanan Venkit, Nan Zhang, Mukund Srinath, Haoran Zhang, Vipul Gupta, Yinghui Li, Tao Li, Fei Wang, Qin Liu, Tianlin Liu, Pengzhi Gao, Congying Xia, Chen Xing, Cheng Jiayang, Zhaowei Wang, Ying Su, Raj Shah, Ruohao Guo, Jing Gu, Haoran Li, Kangda Wei, Zihao Wang, Lu Cheng, Surangika Ranathunga, Meng Fang, Jie Fu, Fei Liu, Ruihong Huang, Eduardo Blanco, Yixin Cao, Rui Zhang, Philip Yu, Wenpeng Yin
| Challenge: | a comparative analysis of paper (meta-)reviews by large language models (LLMs) aims to identify and distinguish LLMs from human activities . |
| Approach: | They present a comparative analysis to identify and distinguish LLM activities from human activities. |
| Outcome: | The proposed analysis aims to improve recognition of instances when someone implicitly uses LLMs for reviewing activities. |
Lunar Twins: We Choose to Go to the Moon with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Lunar Twins is the first LLM designed specifically for lunar exploration . Lunar GenData is a multi-agent collaborative workflow for generating lunar instructions . |
| Approach: | They propose a framework that combines both large and small LLMs and Lunar GenData, which integrates real data from Chang'e lunar missions. |
| Outcome: | Experimental validation shows that the proposed framework enhances domain expertise and reveals indications of embodied intelligence potential. |
Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistency (2024.lrec-main)
Copied to clipboard
| Challenge: | Abstractive summarization models (LLMs) have demonstrated impressive performance in various tasks, but they are still suffering from factual inconsistency problem called hallucination. |
| Approach: | They propose to improve the faithfulness of large language models by impelling them to process the entire article more fairly and faithfully. |
| Outcome: | The proposed strategy improves the faithfulness of large language models in summarization while maintaining their fluency and informativeness. |
SGG-R 3: From Next-Token Prediction to End-to-End Unbiased Scene Graph Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for scene graph generation lack task-specific structured reasoning and sparse, long-tailed relation distributions. |
| Approach: | They propose a structured reasoning framework that integrates task-specific Chain-of-Thought and reinforcement learning with group sequence policy optimization to achieve unbiased scene graph generation. |
| Outcome: | The proposed framework achieves superior performance on two benchmarks. |
Learning to Selectively Learn for Weakly Supervised Paraphrase Generation with Model-based Reinforcement Learning (2022.naacl-main)
Copied to clipboard
| Challenge: | Paraphrase generation is an important natural language generation task . however, the effectiveness of paraphrase generation can be limited due to the limited data available. |
| Approach: | They propose a weakly supervised approach to paraphrase generation that leverages reinforcement learning for effective model training with data selection. |
| Outcome: | The proposed model improves the state-of-the-art performance on four weakly supervised paraphrase generation tasks. |
VIVA+: Human-Centered Situational Decision-Making (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) show promising results in complex, human-centered environments, yet evaluating their capacity for nuanced, humanlike reasoning and decision-making remains challenging. |
| Approach: | They introduce VIVA+, a cognitively grounded benchmark for evaluating the reasoning and decision-making of MLLMs in human-centered situations. |
| Outcome: | The VIVA+ model is based on 1,317 real-world situations paired with 6,373 multiple-choice questions . it consists of three core abilities for decision-making: (1) Foundational Situation Comprehension, (2) Context-Driven Action Justification, and (3) Reflective Reasoning. |
CROP: Zero-shot Cross-lingual Named Entity Recognition with Multilingual Labeled Sequence Translation (2022.findings-emnlp)
Copied to clipboard
Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei
| Challenge: | Named entity recognition (NER) suffers from the scarcity of annotated training data, especially for low-resource languages without labeled data. |
| Approach: | They propose a cross-lingual entity projection framework to enable zero-shot cross-linguistic NER with the help of a multilingual labeled sequence translation model. |
| Outcome: | The proposed method outperforms the baseline method on two benchmarks by a large margin of +3 7 F1 scores and achieves state-of-the-art performance. |
Themis: A Reference-free NLG Evaluation Language Model with Flexibility and Interpretability (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluation of natural language generation tasks lack reliable data. |
| Approach: | They propose to use annotations from human and GPT-4 to construct a corpus for NLG evaluation. |
| Outcome: | The proposed corpus can perform flexible and interpretable evaluations without references and surpasses existing models. |
Improve Language Model and Brain Alignment via Associative Memory (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that associative memory is essential for language comprehension and comprehension. |
| Approach: | They propose to integrate associative memory into language models to improve alignment . they find alignment is improved in brain regions closely related to associativ memory processing . |
| Outcome: | The proposed model improves in brain regions closely related to associative memory processing. |
SAC: Neural Speech Codec with Semantic-Acoustic Dual-Stream Quantization (2026.acl-long)
Copied to clipboard
Wenxi Chen, Ruiqi Yan, Yushen Chen, Zhikang Niu, Ziyang Ma, Xiquan Li, Yuzhe Liang, null Wenhanlin, Shunshun Yin, Ming Tao, Xinsheng Wang, Xie Chen
| Challenge: | Existing speech codecs struggle to balance high-quality reconstruction with semantically rich representations, limiting their effectiveness in both generative and understanding tasks. |
| Approach: | They propose a neural speech codec with semantic-acoustic dual-stream quantization that disentangles semantic and acousian modeling into two dedicated streams. |
| Outcome: | The proposed codec outperforms state-of-the-art speech tokenizers in auto-propagating text-to-speech models. |
CharacterCraft: Bridging the Literature-Reality Dialogue Gap for Practical Role-Playing Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing dialogue datasets have a bias between query distributions and real-world user language usage. |
| Approach: | They propose a framework for Chinese role-playing and a robust evaluation method . they propose specialized Chinese dialogue extraction model and specialized memory retrieval module . |
| Outcome: | The proposed framework extracts character dialogue from novels and ensures high data quality. |
Token Prepending: A Training-Free Approach for Eliciting Better Sentence Embeddings from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on prompt engineering to extract sentence embeddings from large language models (LLMs) but these models are mostly decoder-only and the earlier tokens in the sentence cannot attend to the latter, resulting in biased encoding of sentence information and cascading effects on the final decoded token. |
| Approach: | They propose a plug-and-play and training-free technique that prepends each layer’s decoded sentence embedding to the beginning of the sentence in the next layer’ s input. |
| Outcome: | The proposed technique can significantly improve the performance of existing prompt-based sentence embedding methods across different LLMs while incurring negligible additional inference cost. |
I-AM-G: Interest Augmented Multimodal Generator for Item Personalization (2024.emnlp-main)
Copied to clipboard
| Challenge: | e-commerce and recommender systems lack a framework for personalized generation . a new framework extracts tags from multimodal information of items that the user has interacted with . |
| Approach: | They propose a framework that extracts tags from multimodal information and rewrites item description . they then use a decoupled text-to-text and image-to image retriever to search for similar item text . |
| Outcome: | The proposed framework can generate results aligned with user preferences . it can be used in e-commerce and recommender systems to win over diverse user base . |
Enhancing Retrieval-Augmented Generation via Evidence Tree Search (2025.acl-long)
Copied to clipboard
| Challenge: | Evidence retrieval is used to enhance Large Language Models (LLMs) but in real-world applications, it often returns lengthy documents with redundant or irrelevant content, confusing downstream readers. |
| Approach: | They propose a framework that reformulates evidence retrieval as a dynamic tree expansion process. |
| Outcome: | The proposed framework outperforms existing methods on five datasets. |
DrAgent: Empowering Large Language Models as Medical Agents for Multi-hop Medical Reasoning (2025.findings-emnlp)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xin Liu, Zhengyang Wang, Xianfeng Tang, Shiyang Li, Xiang He, Ruijie Wang, Bing Yin, Xiao Gu, Lei Clifton, David A. Clifton
| Challenge: | commercial LLMs can be difficult to use in real-world clinical decision-making . a lightweight LLM can be used to collaborate with diverse clinical tools . |
| Approach: | They propose a lightweight LLM that can be used to build medical LLMs as agents . they use recursive curriculum learning to optimize the LLM in an easy-to-hard progression . |
| Outcome: | The proposed approach outperforms human experts in medical examinations on diverse datasets. |
Multi-Granularity Optimization for Non-Autoregressive Translation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Non-autoregressive machine translation suffers severe performance deterioration due to the naive independence assumption. |
| Approach: | They propose a method which collects model behaviours on translation segments of various granularities and integrates feedback for backpropagation to reduce latency. |
| Outcome: | Experiments on four benchmark datasets show that the proposed method outperforms baseline models trained with cross-entropy loss and achieves the best performance on WMT’16 EnRo and highly competitive results on WTM’14 EnDe. |
IterAlign: Iterative Constitutional Alignment of Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Empirical results show that iterAlign improves truthfulness, helpfulness, harmlessness and honesty, improving the LLM alignment by up to 13.5% in harmlessness. |
| Approach: | They propose a data-driven constitution discovery and self-alignment framework called IterAlign to overcome these drawbacks by leveraging red teaming to uncover weaknesses of an LLM. |
| Outcome: | Empirical results show that iterAlign improves truthfulness, helpfulness, harmlessness and honesty by up to 13.5%. |
GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that Pre-trained Language Models store relational knowledge and utilize it for performing downstream tasks. |
| Approach: | They propose a benchmark dataset for probing the diversity of relational knowledge in multilingual PLMs. |
| Outcome: | The proposed dataset contains 3125 prompts in English, Chinese, Hindi, Persian, and Swahili . larger multilingual PLMs variants do not store geo-diverse concepts better than its smaller variant . |
Aligning Large Language Models with Implicit Preferences from User-Generated Content (2025.acl-long)
Copied to clipboard
Zhaoxuan Tan, Zheng Li, Tianyi Liu, Haodong Wang, Hyokun Yun, Ming Zeng, Pei Chen, Zhihan Zhang, Yifan Gao, Ruijie Wang, Priyanka Nigam, Bing Yin, Meng Jiang
| Challenge: | Existing preference learning methods rely heavily on curated data from humans or advanced LLMs, which is costly and difficult to scale. |
| Approach: | They propose a framework that leverages implicit preferences in unlabeled user-generated content to generate preference data. |
| Outcome: | The proposed framework transforms user-generated content into user queries and generates responses from the policy model. |
Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewards (2026.acl-long)
Copied to clipboard
Ming Li, Pei Chen, Zhenhao Zhang, Tao Yang, Xinyang Zhang, Han Li, Tianyu Cao, Ming Zeng, Zhuofeng Wu, Meng Jiang, Huasheng Li, Lihong Li, Bing Yin
| Challenge: | Large Language Models exhibit strong capabilities in single-turn instruction following but suffer from Lost-in-Conversation (LiC) when instructions are revealed progressively in multi-turn settings, models get "Lost in Conversation" |
| Approach: | They propose a framework that encourages models to generate correct answers and judge solvability in multi-turn conversations. |
| Outcome: | The proposed framework improves models' ability to balance problem-solving with abstention . it reduces premature answering behaviors that cause lost-in-conversation (LiC) |
Multilingual Agreement for Multilingual Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing models that only use auxiliary languages to encourage multilingual agreement ignore the relationships between different language pairs. |
| Approach: | They propose a multilingual agreement-based method which explicitly models the agreement between different translation directions by randomly substituting some fragments of the source language with their counterpart translations of auxiliary languages. |
| Outcome: | The proposed method improves on the multilingual translation task of 10 language pairs. |
Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training (2025.naacl-long)
Copied to clipboard
Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng, Sanket Lokegaonkar, Yifan Gao, Qing Ping, Tianyi Liu, Binxuan Huang, Zheng Li, Zhengyang Wang, Pei Chen, Ruijie Wang, Rongzhi Zhang, Nasser Zalmout, Priyanka Nigam, Bing Yin, Chao Zhang
| Challenge: | Existing LLMs often rely on complex prompting or extensive fine-tuning to introduce new capabilities while preserving strong generalizability. |
| Approach: | They propose a large-scale pre-training corpus to enhance LLM agents' capabilities . they use 103B agent-specific data encompassing 76,537 APIs . |
| Outcome: | The proposed training corpus outperforms open-source LLMs and commercial LLM agents on three agent benchmarks. |
From Word to World: Can Large Language Models be Implicit Text-based World Models? (2026.acl-long)
Copied to clipboard
Yixia Li, Hongru Wang, Jiahao Qiu, Zhenfei Yin, Dongdong Zhang, Cheng Qian, Zeping Li, Xiaoteng Ma, Guanhua Chen, Heng Ji
| Challenge: | Agentic learning increasingly hinges on interaction, yet real-world experience is expensive, limited, and often irreversible at inference time. |
| Approach: | They propose a framework that reframes language modeling as next-state prediction under interaction. |
| Outcome: | The proposed framework evaluates world models in text-based environments . it shows that sufficiently trained models capture coherent environment dynamics . |
UnClE: Explicitly Leveraging Semantic Similarity to Reduce the Parameters of Word Embeddings (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to reduce word embedding parameters ignore semantic information . existing methods do not consider semantic information, allowing for performance degradation . |
| Approach: | They propose a method that leverages semantic similarity with weight sharing to reduce dimensionality of word embeddings. |
| Outcome: | The proposed method reduces word embedding parameters by more than 11x on a standard English-German dataset. |
Eliciting Implicit Acoustic Styles from Open-domain Instructions to Facilitate Fine-grained Controllable Generation of Speech (2025.emnlp-main)
Copied to clipboard
| Challenge: | Current work relies on pre-defined rules or templates to control the style of speech. |
| Approach: | They propose to use open-domain instructions to generate speech with the acoustic style that meets users’ needs based on their instructions. |
| Outcome: | The proposed model can be used to generate speech with the acoustic style that meets users’ needs based on open-domain instructions. |
Multilingual Knowledge Graph Completion with Self-Supervised Adaptive Graph Alignment (2022.acl-long)
Copied to clipboard
Zijie Huang, Zheng Li, Haoming Jiang, Tianyu Cao, Hanqing Lu, Bing Yin, Karthik Subbian, Yizhou Sun, Wei Wang
| Challenge: | Existing methods to predict missing facts in knowledge graphs are limited in language alignment . SS-AGA uses seed alignment as an edge type to fuses all KGs as a whole graph . |
| Approach: | They propose a self-supervised adaptive graph alignment method that fuses all KGs as a whole graph by regarding alignment as 'a new edge type' they propose SS-AGA method that uses relation-aware attention weights to capture potential alignment pairs in a new paradigm. |
| Outcome: | The proposed method can predict missing facts in a knowledge graph (KG) but language alignment is scarce and new alignment identification is noisy. |
Towards General Agentic Intelligence via Environment Scaling (2026.findings-acl)
Copied to clipboard
Runnan Fang, Shihao Cai, Baixuan Li, Jialong Wu, Guangyu Li, Wenbiao Yin, Xinyu Wang, Xiaobin Wang, Liangcai Su, Zhen Zhang, Shibin Wu, Zhengwei Tao, Yong Jiang, Pengjun Xie, Ningyu Zhang, Fei Huang, Wentao Zhang, Jingren Zhou
| Challenge: | Diverse real-world APIs require precise, robust function-calling intelligence, which needs agents to develop these capabilities through interaction in varied environments. |
| Approach: | They propose a framework that scales up environments to enable agentic intelligence . they use a two-phase agent fine-tuning strategy to first endow agents with basic agentic capabilities, then specializing them for domain-specific contexts. |
| Outcome: | Experiments on -bench, -Bench, and ACEBench show that the model significantly enhances the models’ function-calling capability. |
Knowledge-Selective Pretraining for Attribute Value Extraction (2023.findings-emnlp)
Copied to clipboard
Hui Liu, Qingyu Yin, Zhengyang Wang, Chenwei Zhang, Haoming Jiang, Yifan Gao, Zheng Li, Xian Li, Chao Zhang, Bing Yin, William Wang, Xiaodan Zhu
| Challenge: | Existing methods for AVE are limited on rare attributes due to poor generalization ability. |
| Approach: | They propose to leverage pretraining and transfer learning to address weaknesses in existing methods. |
| Outcome: | The proposed method achieves new state-of-the-art performance without pretraining on rare attributes with limited training resources. |
Xiaomingbot: A Multilingual Robot News Reporter (2020.acl-demos)
Copied to clipboard
Runxin Xu, Jun Cao, Mingxuan Wang, Jiaze Chen, Hao Zhou, Ying Zeng, Yuping Wang, Li Chen, Xiang Yin, Xijin Zhang, Songcheng Jiang, Yuxuan Wang, Lei Li
| Challenge: | Xiaomingbot is a multilingual and multimodal software robot with four capabilities: news generation, news translation, news reading and avatar animation. |
| Approach: | They propose to build a multilingual and multimodal software robot with four inte- gal capabilities: news generation, news translation, news reading and avatar animation. |
| Outcome: | The proposed system generates Chinese news, then reads it in multiple languages and generates an animated avatar reading it. |
SciAssess: Benchmarking LLM Proficiency in Scientific Literature Analysis (2025.findings-naacl)
Copied to clipboard
Hengxing Cai, Xiaochen Cai, Junhan Chang, Sihang Li, Lin Yao, Wang Changxin, Zhifeng Gao, Hongshuai Wang, Li Yongge, Mujie Lin, Shuwen Yang, Jiankun Wang, Mingjun Xu, Jin Huang, Xi Fang, Jiaxi Zhuang, Yuqi Yin, Yaqi Li, Changhong Chen, Zheng Cheng, Zifeng Zhao, Linfeng Zhang, Guolin Ke
| Challenge: | Existing benchmarks fail to adequately evaluate the proficiency of Large Language Models (LLMs) Existing standards do not cover the skills needed to evaluate LLMs in scientific literature analysis. |
| Approach: | They propose a benchmark to evaluate the proficiency of large language models in scientific literature analysis. |
| Outcome: | SciAssess evaluates 11 LLMs on multiple tasks across scientific fields. |
DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can replicate insecure patterns from training data. |
| Approach: | They propose a framework that leverages distributed security-relevant cues by aggregating representations from multiple upper layers via an attention-based module. |
| Outcome: | Experiments show that the framework improves the secure-and-correct generation rate by 11.9% over baselines. |
The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies suggest that strengthening reasoning often coincides with increased hallucination . however, no prior work has examined whether reasoning enhancement itself causes tool hallucinism . |
| Approach: | They propose a diagnostic benchmark measuring tool hallucination in two failure modes . they demonstrate a causal relationship between enhancing reasoning and tool hallubulation . |
| Outcome: | The proposed benchmark measures tool hallucination in two failure modes: no tool available, and (ii) only distractor tools available. |
DisastQA: A Comprehensive Benchmark for Evaluating Question Answering in Disaster Management (2026.findings-acl)
Copied to clipboard
Zhitong Chen, Kai Yin, Xiangjue Dong, Chengkai Liu, Xiangpeng Li, Bo Li, Junwei Ma, Yiming Xiao, Ali Mostafavi, James Caverlee
| Challenge: | Existing benchmarks for question answering (QA) are lacking in a high-stakes environment. |
| Approach: | They propose a rigorously verified benchmark of 3,000 expert-annotated questions . they propose 'keypoint-based evaluation protocol' emphasizing factual completeness over verbosity . |
| Outcome: | Experiments with 20 models reveal substantial divergences from general-purpose models such as MMLU-Pro. |
Recurrent Attention for Neural Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent research questions the importance of dot-product self-attention in Transformer models and shows that most attention heads learn simple positional patterns. |
| Approach: | They propose a novel mechanism to replace dot-product self-attention with a recurrent atteNtion mechanism that directly learns attention weights without token-to-token interaction. |
| Outcome: | The proposed model outperforms the Transformer model on translation tasks with fewer parameters and inference time. |
IHEval: Evaluating Language Models on Following the Instruction Hierarchy (2025.naacl-long)
Copied to clipboard
Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, Meng Jiang
| Challenge: | Instruction-tuned language models (LMs) are increasingly deployed as interactive services across various applications. |
| Approach: | They propose a benchmark to evaluate models' ability to follow the instruction hierarchy by comparing their models to a set of benchmarks. |
| Outcome: | The proposed benchmark covers 3,538 examples across nine tasks covering cases where instructions in different priorities either align or conflict. |
GanLM: Encoder-Decoder Pre-training with an Auxiliary Discriminator (2023.acl-long)
Copied to clipboard
Jian Yang, Shuming Ma, Li Dong, Shaohan Huang, Haoyang Huang, Yuwei Yin, Dongdong Zhang, Liqun Yang, Furu Wei, Zhoujun Li
| Challenge: | Existing pre-training methods underutilize the benefits of language understanding for generation. |
| Approach: | They propose a GAN-style model for encoder-decoder pre-training with an auxiliary discriminator. |
| Outcome: | The proposed model outperforms existing pre-trained models and achieves state-of-the-art performance. |
Instant Personalized Large Language Model Adaptation via Hypernetwork (2026.acl-long)
Copied to clipboard
Zhaoxuan Tan, Zixuan Zhang, Haoyang Wen, Zheng Li, Rongzhi Zhang, Pei Chen, Fengran Mo, Zheyuan Liu, Qingkai Zeng, Qingyu Yin, Meng Jiang
| Challenge: | Existing parameter-efficient fine-tuning methods require training a separate adapter for each user, making them computationally expensive and impractical for real-time updates. |
| Approach: | They propose a scalable framework that maps a user's profile directly to a full set of adapter parameters. |
| Outcome: | The proposed framework outperforms prompt-based personalization and OPPU while using substantially fewer computational resources at deployment. |
Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agents (2026.acl-long)
Copied to clipboard
Zeping Li, Hongru Wang, Yiwen Zhao, Guanhua Chen, Yixia Li, Keyang Chen, Yixin Cao, Guangnan Ye, Hongfeng Chai, Zhenfei Yin
| Challenge: | Large language models excel in mathematical reasoning and multi-hop question answering tasks, but in long trajectories, agents often invoke tools excessively or inappropriately, increasing computation cost and derailing the reasoning process. |
| Approach: | They propose to use entropy reduction as a supervisory signal to reduce tool calls . they propose to design two reward strategies to address the needs of optimizing tool-use behavior. |
| Outcome: | The proposed reward strategies reduce tool calls by 72.07% and improve performance by 22.27%. |
WESR: A Benchmark and Strong Baseline for Word-level Event-Speech Recognition (2026.findings-acl)
Copied to clipboard
Chenchen Yang, Kexin Huang, Liwei Fan, Qian Tu, Botian Jiang, Dong Zhang, Linqi Yin, Shimin Li, Zhaoye Fei, Qinyuan Cheng, Xipeng Qiu
| Challenge: | aaron carroll: the precise localization of non-verbal vocal events remains a critical yet under-explored challenge. carroll says current methods suffer from insufficient task definitions with limited category coverage. carrol: knowing exactly where an event occurred is not enough; knowing exactly what it happened is. |
| Approach: | They propose a taxonomy of 21 vocal events with a new categorization into discrete versus continuous types. |
| Outcome: | The proposed model disentangles ASR errors from event detection while maintaining ASR quality. |
UniConv: Unifying Retrieval and Response Generation for Large Language Models in Conversations (2025.acl-long)
Copied to clipboard
Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, Bing Yin, Meng Jiang
| Challenge: | Existing conversational search systems are usually built with two different models . this separation restricts the system from leveraging the model's intrinsic knowledge simultaneously . Existing studies for developing unified models cannot fully address the aspects of understanding conversational context, managing retrieval independently, and generating responses. |
| Approach: | They propose to unify dense retrieval and response generation for large language models in conversation by fine-tuning and mitigating data discrepancy. |
| Outcome: | The proposed model can outperform existing models on five conversational search datasets and reduce inconsistency risks while mitigating data discrepancy. |
Turn Waste into Worth: Rectifying Top-k Router of MoE (2024.emnlp-main)
Copied to clipboard
Zhiyuan Zeng, Qipeng Guo, Zhaoye Fei, Zhangyue Yin, Yunhua Zhou, Linyang Li, Tianxiang Sun, Hang Yan, Dahua Lin, Xipeng Qiu
| Challenge: | Top-k router suffers from redundancy computation and memory costs due to unbalanced routing . some experts are overflow, where exceeding tokens are dropped, while others are empty, which are padded with zeros, negatively impacting model performance. |
| Approach: | They propose a top-k router that is unbalanced and uses a multi-gPU system to handle dropped tokens and padding. |
| Outcome: | The proposed model surpasses the top-1 router by 4.7% in terms of performance . the top-k router suffers from redundancy computation and memory costs . |
Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models (2026.findings-acl)
Copied to clipboard
Can Xu, Lingyong Yan, Jiayi Wu, Haosen Wang, Shuaiqiang Wang, Yuchen Li, Jizhou Huang, Dawei Yin, Xiang Li
| Challenge: | Existing training paradigms rely on outcome-oriented rewards, which provide insufficient signal for shaping the complex, multi-step reasoning process. |
| Approach: | They propose a framework that integrates large reasoning models with retrieval-augmented generation to improve reasoning fidelity and verification rigor. |
| Outcome: | Experiments on multiple benchmarks demonstrate the effectiveness of the proposed framework. |
MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for query expansion lack corpus-specific knowledge and cost. |
| Approach: | They propose a query-query-document generation method that leverages large language models for mutual verification to produce diverse sub-queries and corresponding documents. |
| Outcome: | The proposed method is fully zero-shot and extensive experiments on three public benchmark datasets demonstrate its effectiveness over existing methods. |
Lost in Literalism: How Supervised Training Shapes Translationese in LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit translationese errors and generate unexpected unnatural translations . Neural machine translation (NMT) has become the dominant method in machine translation research . |
| Approach: | They evaluate the prevalence of translationese in LLM-generated translations and investigate its roots during supervised fine-tuning. |
| Outcome: | The proposed methods reduce translationese while improving translation naturalness . the proposed methods are validated by human evaluations and automatic metrics . |
RACQC: Advanced Retrieval-Augmented Generation for Chinese Query Correction (2025.findings-emnlp)
Copied to clipboard
Jinbo Su, Lingzhe Gao, Wei Li, Shihao Liu, Haojie Lei, Xinyi Wang, Yuanzhao Guo, Ke Wang, Daiting Shi, Dawei Yin
| Challenge: | Large language models (LLMs) exhibit remarkable capabilities across many tasks, but face critical challenges in the CSC scenario: (1) poor generalization to rare entities in open-domain searches; and (2) failure to adapt to temporal entity variations due to static parameters, resulting in serious over-correction issues. |
| Approach: | They propose a Chinese Spelling Check system with RAG and multi-task learning that integrates dynamic knowledge retrieval and entity-centric RAG to address rare entities. |
| Outcome: | The proposed system outperforms existing baselines in the CSC task and achieves a maximum improvement of +9.92% on the search scenario benchmark and +3.2% on the general-domain dataset. |
A Neural Citation Count Prediction Model based on Peer Review Text (D19-1)
Copied to clipboard
| Challenge: | Recent studies have focused on extracting or mining useful features from the paper itself or the associated authors. |
| Approach: | They propose to utilize peer review data for the CCP task with a neural prediction model to learn a comprehensive semantic representation for peer review text. |
| Outcome: | The proposed model improves on the peer review data and hand-crafted features. |
RoleMRC: A Fine-Grained Composite Benchmark for Role-Playing and Instruction-Following (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing role-playing datasets mostly contribute to controlling role style and knowledge boundaries, but overlook role-following in instruction-follower scenarios. |
| Approach: | They propose a fine-grained role-playing and instruction-following composite benchmark, named RoleMRC, which includes multi-turn dialogues between ideal roles and humans, including free chats or discussions upon given passages . |
| Outcome: | The proposed model improves instruction-following without compromising general role-playing and reasoning capabilities. |
KACE: Generating Knowledge Aware Contrastive Explanations for Natural Language Inference (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches in NLP focus on “WHY A” rather than contrastive “WHA NOT B” Experimental results show that contrastive explanations are beneficial to fit the scenarios by clarifying the difference between the predicted answer and other possible wrong ones. |
| Approach: | They propose to generate contrastive explanations with counterfactual examples in NLI by identifying key phrases from input sentences and using them as key perturbations to generate counterfacts. |
| Outcome: | The proposed framework improves on SNLI and ETPA models by 91.9%. |
ReasonRank: Empowering Passage Ranking with Strong Reasoning Ability (2026.acl-long)
Copied to clipboard
| Challenge: | Existing rerankers perform poorly in complex ranking scenarios due to the scarcity of reasoning-intensive training data. |
| Approach: | They propose an automated reasoning-intensive training framework which generates high-quality training labels from training queries and passages. |
| Outcome: | The proposed model outperforms baselines significantly and achieves much lower latency than the pointwise reranker. |
Are Message Passing Neural Networks Really Helpful for Knowledge Graph Completion? (2023.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graphs are far from complete with large portions of triplets missing. |
| Approach: | They propose to use Graph Neural Networks to learn powerful embeddings to improve model performance. |
| Outcome: | The proposed models achieve comparable performance to MLP models, suggesting that MP may not be as crucial as previously thought. |
On Compositional Generalization of Neural Machine Translation (2021.acl-long)
Copied to clipboard
| Challenge: | Modern neural machine translation models have shown competitive performance in benchmarks such as WMT, but there are significant issues such as robustness, domain generalization, etc. |
| Approach: | They propose a benchmark dataset for NMT models from the perspective of compositional generalization and quantitatively analyze the results. |
| Outcome: | The proposed model performs well under traditional metrics, but is low in out-of-domain and low-resource conditions. |
AutoConv: Automatically Generating Information-seeking Conversations with Large Language Models (2023.acl-short)
Copied to clipboard
Siheng Li, Cheng Yang, Yichun Yin, Xinyu Zhu, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang
| Challenge: | Existing research on information-seeking conversations is stymied by the lack of training data. |
| Approach: | They propose to use autoconv for synthetic conversation generation to capture the characteristics of the information-seeking process and fine tune an LLM with a few human conversations to generate synthetic conversations with high quality. |
| Outcome: | The proposed model improves on two commonly-used datasets and alleviates the dependence on human annotation. |
Can Language Models Follow Multiple Turns of Entangled Instructions? (2025.findings-emnlp)
Copied to clipboard
Chi Han, Xin Liu, Haodong Wang, Shiyang Li, Jingfeng Yang, Haoming Jiang, Zhengyang Wang, Qingyu Yin, Liang Qiu, Changlong Yu, Yifan Gao, Zheng Li, Bing Yin, Jingbo Shang, Heng Ji
| Challenge: | Despite of significant achievements in improving instruction-following capabilities of large language models, the ability to process multiple potentially entangled or conflicting instructions remains a considerable challenge. |
| Approach: | They construct multi-turn instruction with 1.1K high-quality multi-turned conversations using the human-in-the-loop approach and examine their capabilities. |
| Outcome: | The proposed model shows that it is difficult to integrate multiple turns and balance competing objectives when instructions intersect or conflict. |
Counterfactual Generator: A Weakly-Supervised Method for Named Entity Recognition (2020.emnlp-main)
Copied to clipboard
| Challenge: | Using labeled data, named entity recognition is labor-intensive, time-consuming and expensive. |
| Approach: | They propose a method which decomposes named entity into two parts: entity and context. |
| Outcome: | The proposed method improves the generalization ability of models under limited observational examples. |
On-policy Reinforcement Fine-tuning with Offline reward for Multi-step Embodied Planning (2026.acl-long)
Copied to clipboard
| Challenge: | Embodied planning requires agents to make coherent multi-step decisions based on dynamic visual observations and verbal goals. |
| Approach: | They propose an On-policy Reinforcement fine-tuning framework with offline rewards for Embodied Task Planning that preserves generalization benefits of RFT while addressing costly interaction and sparse rewards. |
| Outcome: | The proposed framework outperforms closed-source and online-RL methods on EmbodiedBench, a recent benchmark for interactive embodied tasks. |
Tell Me What You Don’t Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editing (2025.findings-acl)
Copied to clipboard
Wenhao Liu, Siyu An, Junru Lu, Muling Wu, Tianlong Li, Xiaohua Wang, Changze Lv, Xiaoqing Zheng, Di Yin, Xing Sun, Xuanjing Huang
| Challenge: | Role-playing Agents (RPAs) struggle to recognize and respond to hard queries that conflict with their role-play knowledge. |
| Approach: | They propose a lightweight representation editing approach that conveniently shifts conflicting requests to the rejection region, thereby enhancing the model’s refusal accuracy. |
| Outcome: | The proposed model improves RPAs’ refusal ability of conflicting requests while maintaining their general role-playing capabilities. |
LongLeader: A Comprehensive Leaderboard for Large Language Models in Long-context Scenarios (2025.naacl-long)
Copied to clipboard
Pei Chen, Hongye Jin, Cheng-Che Lee, Rulin Shao, Jingfeng Yang, Mingyu Zhao, Zhaoyu Zhang, Qin Lu, Kaiwen Men, Ning Xie, Huasheng Li, Bing Yin, Han Li, Lingyun Wang
| Challenge: | LongLeader aims to assess different LLMs' long-context comprehension abilities . long-constext comprehension is a key bottleneck for many use cases . |
| Approach: | They propose a leaderboard to assess different LLMs' long-context comprehension abilities . they offer open-source access to the benchmarks and maintain a dedicated website . |
| Outcome: | The proposed model assesses different LLMs on selected benchmarks and provides open-source access to the benchmarks. |
VideoPro: Adaptive Program Reasoning for Long Video Understanding (2026.acl-long)
Copied to clipboard
Chenglin Li, Feng Han, Yikun Wang, Ruilin Li, Shuai Dong, Haowen Hou, Haitao Li, Qianglong Chen, Feng Tao, Jingqi Tong, Yin Zhang, Jiaqi Wang
| Challenge: | Existing methods for understanding long videos are limited due to the sparsity of visual evidence relevant to a given query. |
| Approach: | They propose a framework that enables VideoLLMs to reason over long videos and refine their predictions through executable programs. |
| Outcome: | The proposed framework outperforms existing methods across long-video understanding benchmarks. |
PRDetect: Perturbation-Robust LLM-generated Text Detection Based on Syntax Tree (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent methods for detecting LLM-generated text have shown impressive performance, but in real-world scenarios, users often introduce perturbations to the text. |
| Approach: | They propose a method that detects syntactic trees that are minimally affected by perturbations and exhibit distinct differences between human-written and LLM-generated text. |
| Outcome: | The proposed method shows that it is significantly better against perturbations on the HC3 and GPT-3.5-mixed datasets and also has the shortest time expenditure. |
Searching for Best Practices in Retrieval-Augmented Generation (2024.emnlp-main)
Copied to clipboard
Xiaohua Wang, Zhenghua Wang, Xuan Gao, Feiran Zhang, Yixin Wu, Zhibo Xu, Tianyuan Shi, Zhengyuan Wang, Shizheng Li, Qi Qian, Ruicheng Yin, Changze Lv, Xiaoqing Zheng, Xuanjing Huang
| Challenge: | Retrieval-augmented generation (RAG) techniques have proven to be effective in integrating up-to-date information, mitigating hallucinations, and enhancing response quality, especially in specialized domains. |
| Approach: | They propose several strategies for deploying RAG that balance performance and efficiency. |
| Outcome: | The proposed approaches can significantly enhance question-answering capabilities and accelerate the generation of multimodal content using a “retrieval as generation” strategy. |
Sense-Aware Neural Models for Pun Location in Texts (P18-2)
Copied to clipboard
| Challenge: | Puns where the two meanings share the same pronunciation are known as homographic puns. |
| Approach: | They propose a sense-aware neural model to address the task of pun location . they first obtain several WSD results for the text and then leverage a bidirectional LSTM network to model each word senses. |
| Outcome: | The proposed model is based on a SemEval 2017 benchmark dataset showing that it can predict homographic puns. |
Theory-optimal Quantization Based on Flatness (2026.acl-long)
Copied to clipboard
| Challenge: | Recent approaches to quantization of Large Language Models (LLMs) have been widely adopted due to activation outliers, which degrade model performance especially at lower bit precision. |
| Approach: | They propose a new metric for quantization that strategically distributes outlier magnitudes across matrix dimensions via optimized diagonal operations. |
| Outcome: | The proposed framework achieves less than 1% accuracy drop in W4A4 quantization on the LLaMA-3-8B model and reduces the performance gap by 39.1% on the more challenging W2A4KV16 model. |
PA-RAG: RAG Alignment via Multi-Perspective Preference Optimization (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing approaches to optimize RAG generators fail to align with RAG requirements thoroughly. |
| Approach: | They propose a method for optimizing the RAG generator from multiple preference perspectives to align with RAG requirements comprehensively. |
| Outcome: | The proposed method improves the performance of RAG generators by incorporating retrieved documents into the prompt. |
Finding RELIEF: Shaping Reasoning Behavior without Reasoning Supervision via Belief Engineering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for shaping large reasoning models rely on reinforcement learning or fine-tuning with gold-standard reasoning traces. Existing techniques for behavior shaping rely only on additional reward modeling. |
| Approach: | They propose a framework that aligns a model's self-concept with a target belief blueprint and internalizes desired traits by fine-tuning on synthesized, self-reflective QA pairs that affirm the target belief. |
| Outcome: | The proposed framework outperforms behavior-supervised and preference-based models while requiring significantly lower training costs. |
Task Knowledge Injection via Interpolations and Reinstatement for Large Language Model Generalization (2025.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained large language models have been widely adopted to elicit their superior performance on downstream tasks, but instruction tuning may overfit them to specific task formats, compromising their generalization on unseen tasks. |
| Approach: | They propose to inject latent task adaptation and knowledge reinstatement into large language models to mitigate spurious correlations between inputs and targets. |
| Outcome: | The proposed method improves generalization on in-domain and out-of-domain unseen tasks. |
CLUE: A Chinese Language Understanding Evaluation Benchmark (2020.coling-main)
Copied to clipboard
Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan
| Challenge: | Existing language evaluation benchmarks for English are limited to English . lack of such benchmarks makes it difficult to replicate success in other languages . |
| Approach: | They introduce a large-scale Chinese language understanding evaluation benchmark . the benchmark uses a set of current state-of-the-art pre-trained Chinese models . |
| Outcome: | The first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark is released . the benchmark evaluates models across a wide range of tasks on original Chinese text . existing language evaluation benchmarks are mostly limited to English . |
ADDMU: Detection of Far-Boundary Adversarial Examples with Data and Model Uncertainty Estimation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods show poor performance under Far Boundary (FB) adversarial examples. |
| Approach: | They propose to use a new technique to detect adversarial examples based on data and model uncertainty to outperform existing methods. |
| Outcome: | The proposed method outperforms existing methods by 3.6 and 6.0 AUC points under each scenario. |
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models (2023.emnlp-main)
Copied to clipboard
Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, Chuanyang Zheng, Xiaodan Liang, Ming Zhang, Qun Liu
| Challenge: | Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning. |
| Approach: | They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
| Outcome: | The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System (2025.acl-long)
Copied to clipboard
Haoyang Su, Renqi Chen, Shixiang Tang, Zhenfei Yin, Xinzhe Zheng, Jinzhe Li, Biqing Qi, Qi Wu, Hui Li, Wanli Ouyang, Philip Torr, Bowen Zhou, Nanqing Dong
| Challenge: | Recent AI methods have shown promise in tasks such as hypothesis generation and experimental design, but they fail to replicate the collaborative nature of real-world scientific practices. |
| Approach: | They propose a virtual scientific system that mimics the collaborative nature of scientific research by organizing a team of agents to generate, evaluate, and refine research ideas. |
| Outcome: | The proposed system outperforms the state-of-the-art method in producing new scientific ideas and offers valuable insights to guide future research. |
Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates (2025.emnlp-main)
Copied to clipboard
Hy Dang, Tianyi Liu, Zhuofeng Wu, Jingfeng Yang, Haoming Jiang, Tao Yang, Pei Chen, Zhengyang Wang, Helen Wang, Huasheng Li, Bing Yin, Meng Jiang
| Challenge: | Large language models (LLMs) have strong reasoning and tool-use capabilities, yet fail in real-world tool-interactions due to incorrect parameterization, poor tool selection, or misinterpretation of user intent. |
| Approach: | They propose a curriculum-inspired framework that leverages structured reasoning templates to guide LLMs through more deliberate step-by-step instructions for generating function calls. |
| Outcome: | The proposed framework reduces tool-use errors and improves interpretability and transparency of tool-using agents. |
Debate-to-Write: A Persona-Driven Multi-Agent Framework for Diverse Argument Generation (2025.coling-main)
Copied to clipboard
| Challenge: | Writing arguments requires integrating high-level beliefs from various perspectives . current language models generate outputs autoregressively, resulting in limited diversity and coherence . |
| Approach: | They propose a persona-based multi-agent framework for argument writing that integrates beliefs from different perspectives into a coherent narrative. |
| Outcome: | The proposed framework generates more diverse arguments by both automatic and human evaluations. |
Learning to Translate by Translating: Stabilizing the Dual Loop via Semantic-Aware Self-Evolution (2026.findings-acl)
Copied to clipboard
Kui Liu, Mingming Yin, Zuoli Tang, Zihao Li, Chilin Fu, Xiaolu Zhang, Jun Zhou, Lixin Zou, Chenliang Li
| Challenge: | Large Language Models (LLMs) have been successful in machine translation, but lack of high-quality parallel corpora and cost constrain scalability. |
| Approach: | They propose an LLM-driven dual-learning framework that enables autonomous translation . they employ a robust semantic-aware reward function that balances adequacy with reconstruction fidelity . |
| Outcome: | The proposed model outperforms larger models on benchmarks and achieves parity with state-of-the-art supervised baselines on mainstream benchmarks. |
ModRWKV: Transformer Multimodality in Linear Time (2025.emnlp-main)
Copied to clipboard
| Challenge: | Currently, multimodal studies are based on large language models with quadratic-complexity Transformer architectures. |
| Approach: | They propose a decoupled multimodal framework built upon the RWKV7 architecture as its LLM backbone and a lightweight architecture to achieve multi-source information fusion. |
| Outcome: | The proposed framework achieves multi-source information fusion through dynamically adaptable heterogeneous modality encoders. |
Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent research focuses on optimizing the use of Self-Docs with their inherent properties remaining underexplored. |
| Approach: | They develop a taxonomy to compare the effectiveness of different types of Self-Docs and explore strategies for combining them with external sources. |
| Outcome: | The proposed model can supplement retrieved content and provide a powerful way to improve knowledge-intensive question answering tasks. |
FormulaReasoning: A Dataset for Formula-Based Numerical Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing datasets for numerical reasoning often lack explicit knowledge of formulas . current datasets do not provide process supervision information, resulting in incomplete reasoning . |
| Approach: | They propose a benchmark for formula-based numerical reasoning with 5,324 questions . they provide annotations in English and Chinese and a formula database as an external knowledge source . |
| Outcome: | The proposed model includes 5,324 questions requiring calculations grounded in external physics principles. |
Improving Consistency for Text Summarization with Energy Functions (2023.findings-emnlp)
Copied to clipboard
Qi Zeng, Qingyu Yin, Zheng Li, Yifan Gao, Sreyashi Nag, Zhengyang Wang, Bing Yin, Heng Ji, Chao Zhang
| Challenge: | Current abstractive summarization models generate inconsistent content due to the inherently noisy dataset and the discrepancy between maximum likelihood estimation based training objectives and consistency measurements. |
| Approach: | They propose a new consistency taxonomy that categorizes inconsistent content into faithfulness, factuality, and self-supportiveness. |
| Outcome: | Experiments on XSUM and CNN/DM datasets show that EnergySum mitigates the trade-off between accuracy and consistency. |
Exploring Memorization in Fine-tuned Language Models (2024.acl-long)
Copied to clipboard
Shenglai Zeng, Yaxin Li, Jie Ren, Yiding Liu, Han Xu, Pengfei He, Yue Xing, Shuaiqiang Wang, Jiliang Tang, Dawei Yin
| Challenge: | Existing studies have shown that pre-trained langauge models tend to memorize and regenerate segments of their pre-training corpus when prompted appropriately. |
| Approach: | They conduct the first comprehensive analysis to explore language models’ memorization during fine-tuning across tasks. |
| Outcome: | The proposed analysis shows that memorization presents a strong disparity among different fine-tuning tasks. |
Pruning Weights but Not Truth: Safeguarding Truthfulness While Pruning LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Neural network pruning disrupts LLMs’ internal activation features crucial for lie detection . layer-wise pruning sparsity inadvertently removes crucial weights, failing to improve lie detection performance despite its reliance on the most crucial LLM layer. |
| Approach: | They propose a pruning approach that places greater emphasis on layers with more activation outliers and stronger discriminative features simultaneously. |
| Outcome: | The proposed approach improves the hallucination detection for pruned LLMs (achieving 88% accuracy at 50% sparsity) and enhances their performance on TruthfulQA. |
CARD: Cross-modal Agent Framework for Generative and Editable Residential Design (2025.emnlp-main)
Copied to clipboard
| Challenge: | Architectural design automation has made significant progress, but the complexity of open-world environments makes residential design a challenging task. |
| Approach: | They propose a framework that leverages a system of specialized cross-modal agents to adapt to open-world residential design. |
| Outcome: | The proposed framework enables users to generate and edit residential design without requiring specialized expertise. |
Agentic-R: Learning to Retrieve for Agentic Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing retrievers for single-turn retrieval-augmented generation (RAG) rely on similarity-based retrievers, but similar passages are not always useful for final answer generation. |
| Approach: | They propose a retrieval-augmented-generation retriever that integrates reasoning with retrieval . they use local query-passage relevance and global answer correctness to measure passage utility . |
| Outcome: | The proposed retriever outperforms existing retrievers on QA benchmarks on seven single-hop and multi-hop searches. |
PychoAgent: Psychology-driven LLM Agents for Explainable Panic Prediction on Social Media during Sudden Disaster Events (2025.emnlp-main)
Copied to clipboard
Mengzhu Liu, Zhengqiu Zhu, Chuan Ai, Chen Gao, Xinghong Li, Lingnan He, Kaisheng Lai, Yingfeng Chen, Xin Lu, Yong Li, Quanjun Yin
| Challenge: | Social media's rich information content and spatiotemporal granularity provide unique opportunities for emotion prediction and management. |
| Approach: | They propose a Psychology-driven generative Agent framework for explainable panic prediction based on emotion arousal theory. |
| Outcome: | The proposed framework improves panic emotion prediction performance by 13% to 21% compared to baseline models. |
Beyond the First Error: Process Reward Models for Reflective Mathematical Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for training effective PRMs focus on the first incorrect step and all preceding steps, assuming that all subsequent steps are incorrect. |
| Approach: | They propose a data annotation method specifically designed to score the long CoT reasoning process by using an LLM-based judger for annotation. |
| Outcome: | The proposed method improves PRMs' ability to identify effective self-correction behaviors and reasoning based on erroneous steps. |
Original Content Is All You Need! an Empirical Study on Leveraging Answer Summary for WikiHowQA Answer Selection Task (2022.coling-1)
Copied to clipboard
Liang Wen, Juan Li, Houfeng Wang, Yingwei Luo, Xiaolin Wang, Xiaodong Zhang, Zhicong Cheng, Dawei Yin
| Challenge: | Existing answer selection approaches for community question answering lack additional answer summaries due to redundancy and lengthiness issues of crowdsourced answers. |
| Approach: | They constructed a dataset which contains a corresponding reference summary for each original lengthy answer. |
| Outcome: | The proposed model improves the performance of a question and candidate answer on a WikiHowQA dataset. |
Evolutionary Contrastive Distillation for Language Model Alignment (2024.findings-emnlp)
Copied to clipboard
Julian Katz-Samuels, Zheng Li, Hyokun Yun, Priyanka Nigam, Yi Xu, Vaclav Petricek, Bing Yin, Trishul Chilimbi
| Challenge: | Existing studies indicate that large language models struggle with challenging instructions. |
| Approach: | They propose a method for generating high-quality synthetic preference data to enhance the complex instruction-following capability of language models. |
| Outcome: | The proposed method exceeds the performance of current SOTA 7B models and is competitive even with open-source 70B models. |
Teaching Small Language Models Reasoning through Counterfactual Distillation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance in a wide range of downstream tasks. |
| Approach: | They propose a counterfactual distillation framework that leverages LLMs to generate high-quality counterfacts and utilizes multi-view CoT to enhance the diversity of reasoning samples. |
| Outcome: | The proposed framework enhances reasoning capabilities of large language models and is more robust to OOD data. |
Reasoning in Flux: Enhancing Large Language Models Reasoning through Uncertainty-aware Adaptive Guidance (2024.acl-long)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Junqi Dai, Qinyuan Cheng, Xuanjing Huang, Xipeng Qiu
| Challenge: | Extensive experiments across various reasoning tasks demonstrate that UAG not only enhances the reasoning abilities of LLMs but consistently outperforms several strong baselines with minimal computational overhead. |
| Approach: | They propose an approach to guide LLMs onto an accurate and reliable trajectory by identifying and adjusting uncertainty signals within each step of the reasoning chain. |
| Outcome: | The proposed approach outperforms strong baselines and outperformed strong models with minimal computational overhead. |
Explainable Quantum Program Repair with Verifiable Proof Traces (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to program repair provide only post-hoc, non-verifiable explanations that are not executable or verifiably. |
| Approach: | They propose a framework that couples repair generation with machine-checkable executable explanations for quantum programs where correctness hinges on subtle semantic properties such as circuit equivalence and fidelity preservation. |
| Outcome: | Experiments on QASMBench with mutation-generated quantum program bugs show that the proposed framework improves both semantic precision and explanation faithfulness over baselines that rely on unconstrained or purely natural-language explanations. |
Multi-Instance Multi-Label Learning Networks for Aspect-Category Sentiment Analysis (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to detect sentiment toward aspect categories ignore the fact that the sentiment of an aspect category mentioned in a sentence is an aggregation of the sentiments of the words indicating the aspect category in the sentence, which leads to suboptimal performance. |
| Approach: | They propose a multi-instance multi-label learning network for Aspect-Category sentiment analysis that treats sentences as bags, words as instances, and the words indicating an aspect category as key instances of the aspect category. |
| Outcome: | The proposed model is based on three public datasets showing that it performs well. |
Multimodal Prompt Learning for Product Title Generation with Extremely Limited Labels (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to generate informative titles for products with limited labels are inadequate for novel products. |
| Approach: | They propose a prompt-based approach to generate attractive titles for novel products . they use multimodal prompts to preserve characteristics and writing styles of novel products. |
| Outcome: | The proposed approach achieves state-of-the-art results on novel product categories with limited labels. |
VIVA: A Benchmark for Vision-Grounded Decision-Making with Human Values (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent large vision language models (VLMs) have demonstrated remarkable intelligence and proficiency across diverse tasks. |
| Approach: | They propose a benchmark for VIsion-grounded decision-making driven by human VA. |
| Outcome: | The proposed model can make decisions under a vision-depicted situation using human values and human values. |
Data Diversity Matters for Robust Instruction Tuning (2024.findings-emnlp)
Copied to clipboard
Alexander Bukharin, Shiyang Li, Zhengyang Wang, Jingfeng Yang, Bing Yin, Xian Li, Chao Zhang, Tuo Zhao, Haoming Jiang
| Challenge: | Recent studies have shown that by curating high quality and diverse instruction tuning datasets, we can significantly improve instruction-following capabilities. |
| Approach: | They propose an algorithm to control diversity and quality of instruction tuning datasets and validate it. |
| Outcome: | The proposed algorithm significantly improves worst and average case performance on large scale instruction tuning datasets. |
Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering (2024.findings-acl)
Copied to clipboard
| Challenge: | Domain-specific question answering (QA) requires a comprehensive understanding of a specific domain to answer specialized questions. |
| Approach: | They propose a new alignment objective to align the LLM preference with different human preferences uniformly to optimize LLM performance in real-world, domain-specific QA settings. |
| Outcome: | The proposed pipeline is superior for real-scenario domain-specific question answering with LLMs. |
Comprehensive Benchmarking of Long-Form Speech Generation in Diverse Scenarios (2026.findings-acl)
Copied to clipboard
Changhao Pan, Rui Yang, Han Wang, Zhuan Zhou, Xuming He, Wenxiang Guo, Ziyue Jiang, Ruiqi Li, Yu Zhang, Chenyuhao Wen, Ke Lei, Xiang Yin, Jingyu Lu, Zhiyuan Zhu, Zhou Zhao
| Challenge: | Existing evaluation benchmarks for long-form speech are limited to limited domains, creating a significant gap with the diverse downstream applications. |
| Approach: | They propose a benchmark that decomposes "long-form speech quality" into specific, disentangled dimensions. |
| Outcome: | The proposed benchmark decomposes “long-form speech quality” into specific, disentangled dimensions. |
Eliminating Biased Length Reliance of Direct Preference Optimization via Down-Sampled KL Divergence (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies attributed verbosity to biased labels, but new research shows that DPO can be effective in mitigating verboses. |
| Approach: | They propose to use a method to reduce the amount of verbosity in LLMs by using a downsampling approach. |
| Outcome: | The proposed approach overcomes the problem of verbosity by reducing the length reliance of the proposed algorithm. |
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts (2023.acl-long)
Copied to clipboard
| Challenge: | Existing research on multi-modal dialogue pre-training is limited due to limited availability of multi-dimensional data . a recent emergence of chatGPT 1 has increased confidence in the potential for this goal . |
| Approach: | They propose a framework for multi-modal dialogue pre-training that integrates experts to accommodate multi-faceted tasks. |
| Outcome: | The proposed framework achieves state-of-the-art on eight multi-modal dialog benchmarks. |
RIPRAG: Hack a Black-box Retrieval-Augmented Generation Question-Answering System with Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to generate RAG documents require knowledge of the target RAG system’s internal composition and implementation details, whereas black-box methods are unable to utilize interactive information. |
| Approach: | They propose a RIPRAG attack framework that treats the target RAG system as a black box and leverages a Reinforcement Learning from Black-box Feedback (RLBF) method to optimize the generation model for poisoned documents. |
| Outcome: | The proposed method achieves an attack success rate (ASR) improvement of up to 0.72 compared to baseline methods. |
CMMaTH: A Chinese Multi-modal Math Skill Evaluation Benchmark for Foundation Models (2025.coling-main)
Copied to clipboard
Zhongzhi Li, Ming-Liang Zhang, Pei-Jie Wang, Jian Xu, Rui-Song Zhang, Yin Fei, Zhi-Long Ji, Jin-Feng Bai, Zhen-Ru Pan, Jiaxin Zhang, Cheng-Lin Liu
| Challenge: | Large language models excel in various language tasks, while large multimodal models effectively handle visual-language problems. |
| Approach: | They propose to use a multimodal multimodal model evaluation benchmark to evaluate model performance in Chinese K12 classrooms. |
| Outcome: | The proposed model evaluation tool is integrated with the CMMaTH dataset. |
MultiConIR: Towards Multi-Condition Information Retrieval (2025.findings-emnlp)
Copied to clipboard
Xuan Lu, Sifan Liu, Bochao Yin, Yongqi Li, Xinghao Chen, Hui Su, Yaohui Jin, Wenjun Zeng, Xiaoyu Shen
| Challenge: | MultiConIR is a benchmark designed to evaluate retrieval and reranking models under nuanced multi-condition query scenarios. |
| Approach: | They propose a benchmark to evaluate retrieval and reranking models under nuanced multi-condition query scenarios. |
| Outcome: | The proposed benchmark evaluates retrieval and reranking models under nuanced multi-condition query scenarios across five domains. |
CityCube: Benchmarking Cross-view Spatial Reasoning on Vision-Language Models in Urban Environments (2026.acl-long)
Copied to clipboard
Haotian Xu, Yue Hu, Zhengqiu Zhu, Chen Gao, Ziyou Wang, Junreng Rao, Wenhao Lu, Weishi Li, Quanjun Yin, Yong Li
| Challenge: | Existing benchmarks focus on indoor or street settings, overlooking challenges of open-ended urban spaces. |
| Approach: | They propose a benchmark to probe cross-view spatial reasoning capabilities of current VLMs in urban settings. |
| Outcome: | The citycube benchmark examines the performance of current vision-language models in urban environments. |
EvolveSearch: An Iterative Self-Evolving Search Agent (2025.emnlp-main)
Copied to clipboard
Ding-Chu Zhang, Yida Zhao, Jialong Wu, Liwen Zhang, Baixuan Li, Wenbiao Yin, Yong Jiang, Yu-Feng Li, Kewei Tu, Pengjun Xie, Fei Huang
| Challenge: | Existing approaches to enabling LLM web search proficiency struggle with data production in open-search domains, while supervised fine-tuning struggles with data utilization efficiency. |
| Approach: | They propose an iterative self-evolution framework that combines SFT and RL to enhance agentic web search capabilities without external human-annotated reasoning data. |
| Outcome: | EvolveSearch achieves 4.7% improvement over current state-of-the-art in seven benchmarks . supervised fine-tuning struggles with data production in open-search domains compared with RL . |
Dynamic Generation of Multi LLM Agents Communication Topologies with Graph Diffusion Models (2026.acl-long)
Copied to clipboard
Eric Hanchen Jiang, Levina Li, Frank Wan, Xiao Liang, Sophia Yin, Yuchen Wu, Xinfeng Li, Yizhou Sun, Wei Wang, Kai-Wei Chang, Ying Nian Wu
| Challenge: | Existing frameworks rely on static or rule-based topologies that fail to adapt to task requirements. |
| Approach: | They propose a generative framework that generates highly task-adaptive topologies . they validated the framework on multiple benchmarks and validated it on multiple platforms . |
| Outcome: | The proposed framework outperforms existing frameworks in task-adaptive communication topologies. |
Memorize Step by Step: Efficient Long-Context Prefilling with Incremental Memory and Decremental Chunk (2024.emnlp-main)
Copied to clipboard
Zhiyuan Zeng, Qipeng Guo, Xiaoran Liu, Zhangyue Yin, Wentao Shu, Mianqiu Huang, Bo Wang, Yunhua Zhou, Linlin Li, Qun Liu, Xipeng Qiu
| Challenge: | Existing methods to optimize LLM for long sequences for long documents are slow and consume memory. |
| Approach: | They propose a method that starts with a small memory size and gradually increases it . they propose Decremental Chunk based on Incremental Memory (IMDC) which reduces chunk size while increasing memory size . |
| Outcome: | The proposed method is faster (1.45x) and reduces GPU memory consumption by 23.3% compared to fixed-size memory. |
Let’s Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts to improve CoT prompting have limitations that require extensive human effort or performance needs to be improved. |
| Approach: | They propose a prompt approach for automatic reasoning called LBS3 inspired by curriculum learning which better reflects human learning habits. |
| Outcome: | The proposed approach achieves strongly competitive performance compared to baselines in reasoning-intensive tasks with varying open- and closed-source LLMs. |
All Information is Valuable: Question Matching over Full Information Transmission Network (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for question matching only transmit one kind of information while failing to utilize both kinds of information simultaneously. |
| Approach: | They propose a question matching network that can transmit both representation and interactive information together in a simultaneous fashion. |
| Outcome: | The proposed approach outperforms strong baseline models on two standard benchmarks. |
DeepPlanner: Scaling Planning Capability for Deep Research Agents via Advantage Shaping (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to planning involve implicit planning or introduce explicit planners without systematically optimizing the planning stage. |
| Approach: | They propose an end-to-end RL framework that enhances the planning capabilities of deep research agents. |
| Outcome: | Experiments show that DeepPlanner improves planning quality and achieves state-of-the-art results under a lower training budget. |
Enhancing Transformers for Generalizable First-Order Logical Entailment (2025.acl-long)
Copied to clipboard
Tianshi Zheng, Jiazheng Wang, Zihao Wang, Jiaxin Bai, Hang Yin, Zheye Deng, Yangqiu Song, Jianxin Li
| Challenge: | Moreover, transformers have demonstrated proficiency in logical reasoning over natural language. |
| Approach: | They propose a logic-aware architecture that improves the performance in generalizable first-order logical entailment by combining distribution shifts and unseen knowledge. |
| Outcome: | The proposed architecture outperforms methods designed specifically for knowledge graph query answering on a dataset with a large dataset. |
MTR-Suite: A Framework for Evaluating and Synthesizing Conversational Retrieval Benchmarks (2026.acl-long)
Copied to clipboard
Junhao Ruan, Abudukeyumu Abudula, Bei Li, Yongjing Yin, Xinyu Liu, Kechen Jiao, Xin Chen, Jingang Wang, Xunliang Cai, Tong Xiao, JingBo Zhu
| Challenge: | Existing conversational retrieval benchmarks suffer from costly, sparse human annotation or rigid, unnatural automated heuristics. |
| Approach: | They propose a framework for auditing, synthesizing, and benchmarking conversational retrieval. |
| Outcome: | The proposed framework is based on three LLM-based auditors and a multi-agent system . it mimics production-style challenges (hard topic switching, verbosity) and offers superior discriminative power. |
Nested Browser-Use Learning for Agentic Information Seeking (2026.acl-long)
Copied to clipboard
Baixuan Li, Jialong Wu, Wenbiao Yin, Kuan Li, Zhongwang Zhang, Huifeng Yin, Zhengwei Tao, Liwen Zhang, Pengjun Xie, Jingren Zhou, Yong Jiang, Wentao Zhang, Zhiqiang Gao
| Challenge: | Existing information-seeking (IS) agents rely on the web for their information acquisition. |
| Approach: | They propose a browser-action framework that decouples interaction control from page exploration through a nested structure. |
| Outcome: | Empirical results show that NestBrowse offers clear benefits in practice. |
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generally, commonsense knowledge is correlated with culture and geographic locations and is only shared locally. |
| Approach: | They construct a Geo-Diverse Visual Commonsense Reasoning dataset to test vision-and-language models’ ability to understand cultural and geo-location-specific commonsense. |
| Outcome: | The proposed models perform better in non-Western regions including East Asia, South Asia, and Africa than in the Western regions. |
Efficient KL Divergence Estimation via Truncated Top-K Integration for Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for estimating KL divergence using only top-k tokens suffer from high variance or systematic bias. |
| Approach: | They propose a top-k Importance-weighted KL Estimator that exploits the Zipfian structure of language model distributions by integrating only the top-K tokens. |
| Outcome: | The proposed estimator outperforms existing estimators on multiple benchmarks while exhibiting lower variance. |
WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, Lihong Li
| Challenge: | Existing work on reinforcement learning has focused on single-turn tasks such as solving math problems. |
| Approach: | They propose a framework that learns directly from online interactions by asynchronously generating diverse trajectories, guided by binary rewards depending on task success. |
| Outcome: | Experiments on the WebArena-Lite benchmark show that the framework outperforms state-of-the-art methods and strong proprietary models. |
Ultra-fine Entity Typing with Indirect Supervision from Natural Language Inference (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing methods for ultra-fine entity typing fail to capture type semantics because of the large number of types and the scarcity of data per type. |
| Approach: | They propose a method that formulates entity typing as a natural language inference problem . they use indirect supervision from NLI to infer type information as textual hypotheses . |
| Outcome: | The proposed method achieves state-of-the-art performance on the ultra-fine entity typing task with limited training data. |
Language Reconstruction with Brain Predictive Coding from fMRI Data (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that the perception of speech can be decoded from brain signals and subsequently reconstructed as continuous language. |
| Approach: | They propose to use FMRI-to-text decoding with Predictive coding to generate a main network and a side network to generate brain predictive representations from related regions of interest. |
| Outcome: | The proposed model outperforms current decoding models on several evaluation metrics on two naturalistic language comprehension fMRI datasets. |
DORM: Preference Data Weights Optimization for Reward Modeling in LLM Alignment (2025.findings-emnlp)
Copied to clipboard
Rongzhi Zhang, Chenwei Zhang, Xinyang Zhang, Liang Qiu, Haoming Jiang, Yuchen Zhuang, Qingru Zhang, Hyokun Yun, Xian Li, Bing Yin, Tuo Zhao, Chao Zhang
| Challenge: | Existing approaches to align large language models with human preferences are noisy and varying in importance of preference samples. |
| Approach: | a new method enhances reward modeling by learning to dynamically weigh preference data. |
| Outcome: | a new method improves the performance of large language models with human preferences . it initializes data importance and iteratively refines them to maximize validation performance. |
MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark (2025.acl-long)
Copied to clipboard
Qihao Zhao, Yangyu Huang, Tengchao Lv, Lei Cui, Qinzheng Sun, Shaoguang Mao, Xin Zhang, Ying Xin, Qiufeng Yin, Scarlett Li, Furu Wei
| Challenge: | Multiple-choice question datasets like Massive Multitask Language Understanding (MMLU) have inevitably led to benchmark contamination, resulting in unreliable evaluation. |
| Approach: | They propose a contamination-free MCQ benchmark called MMLU-CF which reassesses LLMs’ understanding of world knowledge by averting both unintentional and malicious data contamination. |
| Outcome: | The proposed MMLU-CF reassesses LLMs’ understanding of world knowledge by averting both unintentional and malicious data contamination. |
MetaTS: Meta Teacher-Student Network for Multilingual Sequence Labeling with Minimal Supervision (2021.emnlp-main)
Copied to clipboard
| Challenge: | Sequence labeling aims to predict fine-grained sequences of labels for text, but lack of token-level annotated data hinders the effectiveness of supervised methods. |
| Approach: | They propose a Meta Teacher-Student (MetaTS) Network to alleviate data scarcity by leveraging large multilingual unlabeled data. |
| Outcome: | The proposed meta learning method alleviates data scarcity by leveraging large multilingual unlabeled data. |
DT-Solver: Automated Theorem Proving with Dynamic-Tree Sampling Guided by Proof-level Value Function (2023.acl-long)
Copied to clipboard
Haiming Wang, Ye Yuan, Zhengying Liu, Jianhao Shen, Yichun Yin, Jing Xiong, Enze Xie, Han Shi, Yujun Li, Lin Li, Jian Yin, Zhenguo Li, Xiaodan Liang
| Challenge: | Recent advances in neural theorem-proving resort to large language models and tree searches. |
| Approach: | They propose a Dynamic-Tree Driven Theorem Solver to accommodate general theoremes by guiding the search procedure with state confidence and proof-level values. |
| Outcome: | The proposed method outperforms state-of-the-art methods on two popular theorem-proving datasets with a 6.65% improvement on average in terms of success rate. |
Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning (2024.emnlp-main)
Copied to clipboard
Abhinav Bandari, Lu Yin, Cheng-Yu Hsieh, Ajay Jaiswal, Tianlong Chen, Li Shen, Ranjay Krishna, Shiwei Liu
| Challenge: | Existing approaches to prune LLMs rely on the C4 dataset as calibration data . arithmetic datasets perform better than pre-training datasets for pruning, whereas chain-of-thought is only useful on certain tasks. |
| Approach: | They evaluate the selection of calibration data for LLM pruning across a wide range of datasets . they find that C4 is not the optimal calibration data, and that CoT is only useful on certain tasks. |
| Outcome: | The chosen calibration data significantly impacts the performance of pruned LLMs, the authors found . their results shed light on the importance of carefully selecting calibration data for LLM pruning . |
Generate & Rank: A Multi-task Framework for Math Word Problems (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies formalize MWP as a generation task but mathematical expressions are prone to minor mistakes. |
| Approach: | They propose a ranking task for math word problem (MWP) that learns from its own mistakes and distinguishes between correct and incorrect expressions. |
| Outcome: | The proposed model outperforms baselines on the classical Math23k dataset and is 7% higher than the state-of-the-art. |
Enhancing Reinforcement Learning with Label-Sensitive Reward for Natural Language Understanding (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have yielded remarkable performance, but objective mismatch issues hinder RLHF learning. |
| Approach: | They propose a Reinforcement Learning framework enhanced with Label-sensitive reward to enhance LLMs' alignment and generation capabilities. |
| Outcome: | The proposed framework improves performance on five diverse models across eight tasks. |
BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Search (2026.findings-acl)
Copied to clipboard
Shiyu Liu, Yongjing Yin, Jianhao Yan, Yunbo Tang, Qinggang Zhang, Bei Li, Xin Chen, Jingang Wang, Xunliang Cai, Jinsong Su
| Challenge: | Existing RL-based agentic search models fail to recognize reasoning boundaries and rarely admit "I DON'T KNOW" lack of reliability leads to plausible but unreliable answers, introducing significant risks . |
| Approach: | They propose a framework to cultivate reliable boundary awareness without compromising accuracy. |
| Outcome: | Experiments show that the proposed framework improves the reliability of agentic search models. |
VehicleWorld: A Highly Integrated Multi-Device Environment for Intelligent Vehicle Interaction (2025.findings-emnlp)
Copied to clipboard
Jie Yang, Jiajun Chen, Zhangyue Yin, Shuo Chen, Yuxin Wang, Yiran Guo, Yuan Li, Yining Zheng, Xuanjing Huang, Xipeng Qiu
| Challenge: | Traditional Function Calling (FC) approaches operate statelessly, requiring multiple exploratory calls to build environmental awareness before execution, leading to inefficiency and limited error recovery. |
| Approach: | They propose a state-based function call approach that maintains explicit system state awareness and implements direct state transitions to achieve target conditions. |
| Outcome: | The proposed approach outperforms traditional function calling approaches, achieving superior execution accuracy and reduced latency. |
LANS: A Layout-Aware Neural Solver for Plane Geometry Problem (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing neural solvers take GPS as vision-language task but lack layout awareness . Existing models are criticized for complex rules and poor adaptability . |
| Approach: | They propose a layout-aware neural solver called LANS that integrates two modules to solve GPS. |
| Outcome: | The proposed solver outperforms existing neural and symbolic solvers on two datasets. |
T2S-GPT: Dynamic Vector Quantization for Autoregressive Sign Language Production from Text (2024.acl-long)
Copied to clipboard
| Challenge: | Existing vector quantization methods are fixed-length encodings, overlooking the uneven information density in sign language. |
| Approach: | They propose a two-stage sign language production paradigm that encodes sign language sequences into discrete codes and autoregressively generates sign languages from text. |
| Outcome: | The proposed model can dynamically adjust the encoding length based on the information density in sign language to achieve accurate and compact encoded enccoding. |
Rethinking Cross-Subject Data Splitting for Brain-to-Text Decoding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have successfully decoded natural language from non-invasive brain signals . current dataset splitting methods suffer from data leakage problem . |
| Approach: | They propose a right cross-subject data splitting criterion without data leakage for decoding fMRI and EEG signal to text. |
| Outcome: | The proposed method overfits and overestimates brain-to-text decoding models. |
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation (2024.findings-acl)
Copied to clipboard
Xize Cheng, Rongjie Huang, Linjun Li, Zehan Wang, Tao Jin, Aoxiong Yin, Chen Feiyang, Xinyu Duan, Baoxing Huai, Zhou Zhao
| Challenge: | Existing methods for talking head translation rely on cascading, resulting in delays and cascadic errors. |
| Approach: | They propose a model for talking head translation, TransFace, which can translate audio-visual speech into audio-visual speech in other languages. |
| Outcome: | The proposed model can translate audio-visual speech into audio-visual speech in other languages. |
ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilities (2025.findings-naacl)
Copied to clipboard
Jiarui Lu, Thomas Holleis, Yizhe Zhang, Bernhard Aumayer, Feng Nan, Haoping Bai, Shuang Ma, Shen Ma, Mengyu Li, Guoli Yin, Zirui Wang, Ruoming Pang
| Challenge: | Recent advances in large language models have led to a growing interest in tool assisted LLMs . toolSandbox includes stateful tool execution, implicit state dependencies between tools . |
| Approach: | a new tool-based evaluation tool is released to help LLMs evaluate their tool-use capabilities. a tool-driven evaluation tool includes stateful tool execution, implicit state dependencies between tools and a built-in user simulator. |
| Outcome: | the toolSandbox evaluation benchmark shows that open source and proprietary models have a performance gap . the benchmarks show that even the most capable LLMs are challenged by state dependent tasks . |
Natural Language to Code Generation in Interactive Data Science Notebooks (2023.acl-long)
Copied to clipboard
Pengcheng Yin, Wen-Ding Li, Kefan Xiao, Abhishek Rao, Yeming Wen, Kensen Shi, Joshua Howland, Paige Bailey, Michele Catasta, Henryk Michalewski, Oleksandr Polozov, Charles Sutton
| Challenge: | Data scientists use computational notebooks to perform data wrangling and analytic tasks. |
| Approach: | They build a benchmark program that synthesizes programs given NL intents from users by using a Python code language model. |
| Outcome: | The proposed model outperforms public code LMs in a dataset of 1078 code generation problems using the pandas data analysis framework in data science notebooks. |
Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Extensive research has highlighted the quality of instruction data is essential for the success of this alignment. |
| Approach: | They propose a framework for iteratively improving existing instruction data by using Monte Carlo tree search to find suitable prompts that align the language model to effectively learn multiple skills. |
| Outcome: | The proposed framework improves the evaluation scores of seed instruction data, raising the average evaluation scores from 2.19 to 3.81. |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
Chain-Talker: Chain Understanding and Rendering for Empathetic Conversational Speech Synthesis (2025.findings-acl)
Copied to clipboard
| Challenge: | Current generative CSS models face interpretability limitations due to insufficient emotional perception and redundant discrete speech coding. |
| Approach: | They propose a framework that aligns synthesized speech with the emotional context of user-agent interactions to achieve empathy. |
| Outcome: | The proposed framework produces more expressive speech than existing methods on three datasets. |
Beyond Completion: A Foundation Model for General Knowledge Graph Reasoning (2025.findings-acl)
Copied to clipboard
Yin Hua, Zhiqiang Liu, Mingyang Chen, Zheng Fang, Chi Man Wong, Lingxiao Li, Chi Man Vong, Huajun Chen, Wen Zhang
| Challenge: | Existing foundation models for general knowledge graph reasoning have focused on their structural aspects, with most efforts restricted to in-KG tasks. |
| Approach: | They propose a conditional encoding architecture that bridges the gap between textual and structural modalities, enabling seamless integration. |
| Outcome: | The proposed model outperforms baseline models on 28 datasets and is generalized to out-of-KG tasks. |
Sequential-NIAH: A Needle-In-A-Haystack Benchmark for Extracting Sequential Needles from Long Contexts (2025.emnlp-main)
Copied to clipboard
Yifei Yu, Qian-Wen Zhang, Lingfeng Qiao, Di Yin, Fang Li, Jie Wang, Chen Zeng Xi, Suncong Zheng, Xiaolong Liang, Xing Sun
| Challenge: | Recent models have extended Corresponding Author. context lengths to millions of tokens while maintaining reasoning and comprehension capabilities. |
| Approach: | They propose a benchmark to evaluate the ability of large language models to extract sequential information items from long contexts. |
| Outcome: | The proposed model achieves maximum accuracy of 63.50% on six well-known LLMs. |
ReSo: A Reward-driven Self-organizing LLM-based Multi-Agent System for Reasoning Tasks (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multi-agent systems (MAS) are limited by poor flexibility and scalability, with underdeveloped optimization strategies. |
| Approach: | They propose a task graph generation and a reward-driven two-stage agent selection process to integrate multi-agent systems to improve their reasoning capabilities. |
| Outcome: | The proposed model outperforms existing methods on Math-MAS and SciBench-MAS SciBech, while other methods completely fail. |
Automatic Prompt Optimization with “Gradient Descent” and Beam Search (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive performance as general purpose agents, but their abilities remain highly dependent on prompts which are hand written with onerous trial-and-error effort. |
| Approach: | They propose an algorithm that uses numerical gradient descent to automatically improve prompts by rewriting vague task descriptions into more precise annotation instructions. |
| Outcome: | The proposed algorithm outperforms previous methods and improves performance on three benchmark NLP tasks and the novel problem of LLM jailbreak detection. |
Beyond Dialogue Time: Temporal Semantic Memory for Personalized LLM Agents (2026.findings-acl)
Copied to clipboard
Miao Su, Yucan Guo, Zhongni Hou, Long Bai, Zixuan Li, Yufei Zhang, Guojun Yin, Wei Lin, Xiaolong Jin, Jiafeng Guo, Xueqi Cheng
| Challenge: | Existing methods focus on point-wise memory, losing durative information that captures persistent states and evolving patterns. |
| Approach: | They propose a memory framework that models semantic time for point-wise memory and supports the construction and utilization of durative memory. |
| Outcome: | Experiments on LongMemEval and LoCoMo show that the proposed method outperforms existing methods and achieves up to 12.2% improvement in accuracy. |
Red Teaming Large Reasoning Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) have emerged as a powerful advancement in multi-step reasoning tasks, but they introduce safety and reliability risks, such as CoT-hijacking and prompt-induced inefficiencies. |
| Approach: | They propose a unified benchmark to assess the trustworthiness of Large Reasoning Models. |
| Outcome: | The proposed benchmark evaluates truthfulness, safety and efficiency on 26 models. |
R3-RAG: Learning Step-by-Step Reasoning and Retrieval for LLMs via Reinforcement Learning (2025.findings-emnlp)
Copied to clipboard
Yuan Li, Qi Luo, Xiaonan Li, Bufan Li, Qinyuan Cheng, Bo Wang, Yining Zheng, Yuxin Wang, Zhangyue Yin, Xipeng Qiu
| Challenge: | RAG systems that integrate external knowledge with Large Language Models often become bottlenecks due to their limited parameters compared to LLMs and their inability to perform step-by-step reasoning. |
| Approach: | They propose a model that integrates external knowledge with Large Language Models to enhance factual correctness and mitigate hallucination. |
| Outcome: | The proposed model outperforms baselines and can transfer well to different retrievers. |
Categorizing Semantic Representations for Neural Machine Translation (2022.coling-1)
Copied to clipboard
| Challenge: | Modern neural machine translation models suffer limitation in compositional generalization, resulting in weakened translation performance on unseen compounds. |
| Approach: | They propose to introduce categorization to the contextualized representations to improve generalization by reducing sparsity and overfitting. |
| Outcome: | The proposed method reduces compositional generalization error rates by 24% on a dedicated MT dataset. |
Decoding the Echoes of Vision from fMRI: Memory Disentangling for Past Semantic Information (2024.emnlp-main)
Copied to clipboard
| Challenge: | Experimental results demonstrate that this method effectively disentangles the information within fMRI signals. |
| Approach: | They propose a task Memory Disentangling which extracts and decodes past information from fMRI signals. |
| Outcome: | The proposed method extracts and decodes past information from fMRI signals. |
Exploring the Cost-Effectiveness of Perspective Taking in Crowdsourcing Subjective Assessment: A Case Study of Toxicity Detection (2025.naacl-long)
Copied to clipboard
| Challenge: | toxicity evaluation tasks require annotations to accurately reflect opinions of subgroups . toxicity tasks require annotators to take the opinions of a subgroup simultaneously . |
| Approach: | They propose to use perspective taking to obtain opinions from subgroups . they propose to prompt annotators to take perspectives of contrasting subgroup simultaneously . |
| Outcome: | The proposed approach can be cost-effective and improve quality under limited budget. |
m3P: Towards Multimodal Multilingual Translation with Multimodal Prompt (2024.lrec-main)
Copied to clipboard
Jian Yang, Hongcheng Guo, Yuwei Yin, Jiaqi Bai, Bing Wang, Jiaheng Liu, Xinnian Liang, LinZheng Chai, Liqun Yang, Zhoujun Li
| Challenge: | Existing multimodal neural machine translation models focus on bilingual translation, but experimental results show that they outperform the text-only baselines and multilingual multimodal methods by a large margin. |
| Approach: | They propose a framework to leverage the multimodal prompt to guide the Multimodal Multilingual Neural Machine Translation (m3P) this framework aligns the representations of different languages with the same meaning and generates the conditional vision-language memory for translation. |
| Outcome: | The proposed framework outperforms previous text-only baselines and multilingual multimodal methods by a large margin. |
Show Me More Details: Discovering Hierarchies of Procedures from Semi-structured Web Data (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work has treated procedures as shallow structures without modeling the parent-child relation. |
| Approach: | They propose to construct an open-domain hierarchical knowledge-base (KB) of procedures based on wikiHow . they link steps in an article to other articles with similar goals, recursively building the KB . |
| Outcome: | The proposed method significantly outperforms baselines according to automatic evaluation, human judgment, and application to downstream tasks such as instructional video retrieval. |
Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models (2024.acl-long)
Copied to clipboard
Zihao Lin, Mohammad Beigi, Hongxuan Li, Yufan Zhou, Yuxiang Zhang, Qifan Wang, Wenpeng Yin, Lifu Huang
| Challenge: | Memory Editing (ME) has emerged as an efficient method to modify erroneous facts or inject new knowledge into Large Language Models (LLMs). |
| Approach: | They propose to evaluate LLMs with single edit only and parameter-modifying ME with parameter-preserving ME. |
| Outcome: | The proposed method can maintain LLMs’ fundamental capabilities but struggles to accurately recall edited knowledge presented in a different format. |
UniCoder: Scaling Code Large Language Model via Universal Code (2024.acl-long)
Copied to clipboard
Tao Sun, Linzheng Chai, Jian Yang, Yuwei Yin, Hongcheng Guo, Jiaheng Liu, Bing Wang, Liqun Yang, Zhoujun Li
| Challenge: | Experimental results show that UniCoder with the universal code significantly outperforms the previous prompting methods by a large margin. |
| Approach: | They introduce the universal code (UniCode) as the intermediate representation of algorithm steps using conventions of programming languages. |
| Outcome: | The proposed model outperforms previous prompting methods by a large margin . the proposed model is based on a dataset of natural-language questions and code solutions . |
Red Teaming Visual Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | VLMs (Vision-Language Models) can be induced to generate harmful or inaccurate content through specific test cases. |
| Approach: | They propose a red teaming dataset which encompasses 12 subtasks under 4 primary aspects (faithfulness, privacy, safety, fairness) this dataset is the first to benchmark current VLMs in terms of these 4 aspects . |
| Outcome: | The proposed dataset shows that 10 open-source VLMs struggle with red teaming in different degrees and have up to 31% performance gap with GPT-4V. |
ConTinTin: Continual Learning from Task Instructions (2022.acl-long)
Copied to clipboard
| Challenge: | a new learning paradigm is proposed for NLP, which seeks supervision for solving a target task. |
| Approach: | They propose a new learning paradigm that uses textual instructions to learn new tasks . the main goal of machine learning algorithms lies in seeking supervision for solving a target task. |
| Outcome: | The proposed learning paradigm is based on a stream of more than 60 tasks . it makes full use of task instructions to improve forward-transfer and backward-transference . |
Contextual Modeling for Document-level ASR Error Correction (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing work on document-level ASR error correction ignores contextual information . however, there are limited studies on incorporating contextual information into AEC . |
| Approach: | They propose a context-aware method that retrieves contextual information from a datastore . they use two English and two Chinese datasets to model document-level AEC . |
| Outcome: | The proposed model can utilize contextual information to improve document-level AEC . the data store containing contextual information provides even better results . |
DORA: A Dual-Objective Reinforcement Learning Framework for Effective and Efficient Multimodal Agentic Search (2026.acl-long)
Copied to clipboard
Guangming Qin, Yuhao Deng, Yukun Zhao, Zhenyang Li, Junfeng Wang, Dawei Yin, Ye Yuan, Guoren Wang, Yizhou Yan, Chengliang Chai, Lei Cao
| Challenge: | Existing methods to train large language models overlook quality of intermediate search results . existing methods often invoke search calls during reasoning, making inference inefficient . |
| Approach: | They propose a dual-objective reinforcement learning framework to improve search strategies of MLLMs . DORA outperforms state-of-the-art methods, achieving up to 8.4% higher accuracy . |
| Outcome: | The proposed model outperforms state-of-the-art methods while reducing search calls by 9.7%. |
OS-Sentinel: Towards Safety-Enhanced Mobile GUI Agents via Hybrid Validation in Realistic Workflows (2026.acl-long)
Copied to clipboard
Qiushi Sun, Mukai Li, Zhoumianze Liu, Zhihui Xie, Fangzhi Xu, Zhangyue Yin, Kanzhi Cheng, Zehao Li, Zichen Ding, Qi Liu, Zhiyong Wu, Zhuosheng Zhang, Ben Kao, Lingpeng Kong
| Challenge: | Existing methods for detecting unsafe mobile GUI agents are underexplored. |
| Approach: | They propose a mobile agent safety detection framework that integrates a formal verifier and a VLM-based contextual judge to detect system-level violations. |
| Outcome: | The proposed framework achieves 10%–30% improvements over existing approaches across multiple metrics. |
Cultural Concept Adaptation on Multimodal Reasoning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Past methods focused on multilingual and multimodal capabilities, and the improvement of multicultural competence is still an unexplored problem. |
| Approach: | They propose an annotation-free method for cultural-concept adaptation and construct a concept mapping set to facilitate model's comprehension of cultural-consensual mappings. |
| Outcome: | The proposed method outperforms baseline models on zero-shot and few-shot settings on five languages and cultures. |
Task-guided Disentangled Tuning for Pretrained Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Pretrained language models are fine-tuned on task-specific datasets, but fail to capture task- specific patterns. |
| Approach: | They propose a method which disentangles task-relevant signals from entangled representations. |
| Outcome: | The proposed method improves generalization of representations by disentangling task-relevant signals from the entangled representations. |
A Novel Global Feature-Oriented Relational Triple Extraction Model based on Table Filling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Table filling based relational triple extraction methods focus on using local features but ignore the global associations of relations and token pairs, which increases the possibility of overlooking some important information during triple extraction. |
| Approach: | They propose a global feature-oriented triple extraction model that makes full use of the two kinds of global associations of relations and token pairs. |
| Outcome: | The proposed model achieves state-of-the-art on three benchmark datasets. |
Towards Autonomous Tool Utilization in Language Models: A Unified, Efficient and Scalable Framework (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in tool learning for large language models have led to a new trend to allow LLMs to leverage external tools. |
| Approach: | They propose a framework for fine-tuning language models that categorizes queries into three different types . they also introduce an "instruct, execute, and reformat" strategy specifically designed for efficient data annotation . |
| Outcome: | The proposed framework surpasses open-source language models and GPT-3.5/4 on multiple evaluation metrics. |
Explicit Syntactic Guidance for Neural Text Generation (2023.acl-long)
Copied to clipboard
| Challenge: | Existing text generation models follow the sequence-to-sequence paradigm . generative grammar suggests humans generate language by learning language grammar . |
| Approach: | They propose a syntax-guided generation schema that searches the syntax tree in a top-down direction. |
| Outcome: | The proposed method outperforms autoregressive baselines on paraphrase generation and machine translation. |
ProCQA: A Large-scale Community-based Programming Question Answering Dataset for Code Search (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to code question answering use bi-modal and unimodal pretraining to align text and code representations. |
| Approach: | They propose a modality-agnostic contrastive pre-training approach to improve alignment of text and code representations of current code language models. |
| Outcome: | The proposed model exhibits significant performance improvements across a wide range of code retrieval benchmarks. |
What Have We Achieved on Non-autoregressive Translation? (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies have shown that non-autoregressive (NAT) methods underperform autoregressive methods (AT) however, their evaluation using BLEU has been shown to weakly correlate with human annotations. |
| Approach: | They propose to evaluate four representative NAT methods using BLEU to narrow the performance gap between autoregressive and autoregressive translations. |
| Outcome: | The proposed methods underperform NAT and autoregressive methods under more reliable evaluation metrics. |
Co-Evolving LLMs and Embedding Models via Density-Guided Preference Optimization for Text Clustering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for text clustering use static pseudo-oracles, i.e., unidirectionally querying them for similarity assessment or data augmentation. |
| Approach: | They propose a training framework that enables bidirectional refinement between LLMs and embedding models by using task-aware prompts to guide the LLM in generating interpretations for the input texts. |
| Outcome: | Experiments on 14 benchmark datasets across 5 tasks demonstrate the effectiveness of the proposed training framework. |
Strengthened Symbol Binding Makes Large Language Models Reliable Multiple-Choice Selectors (2024.acl-long)
Copied to clipboard
| Challenge: | Multiple-Choice Questions (MCQs) are a critical area of research in the study of Large Language models (LLMs). |
| Approach: | They propose an efficient SFT algorithm for MCQs, termed Point-wise Intelligent Feedback, which constructs negative instances by randomly combing the incorrect option contents with all candidate symbols. |
| Outcome: | The proposed algorithm significantly reduces the model’s selection bias by improving its MCSB capability. |
Thermometer of Thoughts: Enhancing LLM’s Exploration via Attention Temperature Modulation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in the reasoning capabilities of large language models have enabled them to tackle complex tasks such as mathematics reasoning. |
| Approach: | They propose a method that modulates attention temperature dynamically to shift LLM's internal focus during reasoning, enabling a dynamic shift between exploratory and focused modes. |
| Outcome: | The proposed method improves Pass@10 by 6.78–14.20% and aggregation accuracy by 9.74% across 7 reasoning benchmarks. |
LexMatcher: Dictionary-centric Data Curation for LLM-based Machine Translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | emergence of large language models (LLMs) has brought about new opportunities for machine translation. |
| Approach: | They propose a method for data curation that supplements the infrequent senses of polysemous words. |
| Outcome: | The proposed method outperforms established baselines on the WMT2022 test sets and is applicable to other pre-trained models. |
GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) and multi-modal models (MMs) have demonstrated remarkable capabilities in problem-solving, but their proficiency in tackling geometry math problems has not been thoroughly evaluated. |
| Approach: | They propose a benchmark to evaluate the performance of large language models and multi-modal models in solving geometry math problems. |
| Outcome: | The proposed model achieves 55.67% accuracy on main subset but only 6.00% accuracy on hard subset. |
MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding (2024.emnlp-main)
Copied to clipboard
Baixuan Xu, Weiqi Wang, Haochen Shi, Wenxuan Ding, Huihao Jing, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Long Chen, Yangqiu Song
| Challenge: | Existing methods for acquiring large-scale intentions generate product-centric intentions without product images and incur high costs for scalability. |
| Approach: | They propose a multimodal framework that allows Large Vision-Language Models to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. |
| Outcome: | The proposed framework shows that it is robust to different prompts and superior to previous methods. |
UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models (2022.emnlp-main)
Copied to clipboard
Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zettlemoyer, Tao Yu
| Challenge: | Structured knowledge grounding (SKG) uses structured knowledge to complete user requests . since inputs and outputs of SKG tasks are heterogeneous, they have been studied separately . |
| Approach: | They propose a framework that unifies 21 SKG tasks into a text-to-text format . they use unifiedSKG to benchmark T5 with different sizes . |
| Outcome: | The proposed framework unifies 21 SKG tasks into a text-to-text format . it achieves state-of-the-art performance on almost all of the 21 tasks, the authors show . |
Stratagem: Learning Transferable Reasoning via Trajectory-Modulated Game Self-Play (2026.acl-long)
Copied to clipboard
Xiachong Feng, Deyi Yin, Xiaocheng Feng, Yi Jiang, Libo Qin, Yangfan Ye, Lei Huang, Weitao Ma, Qiming Li, Yuxuan Gu, Bing Qin, Lingpeng Kong
| Challenge: | Existing self-play approaches to developing general reasoning in language models rely on terminal game outcomes. |
| Approach: | They propose a game-based reasoning transfer model that addresses two barriers to reasoning transfer. |
| Outcome: | The proposed model improves mathematical reasoning, general reasoning, and code generation benchmarks. |
Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors (2026.acl-long)
Copied to clipboard
Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, Shouling Ji
| Challenge: | Existing methods to inject safety-aligned large language models rely on token-level mappings, which do not guarantee sustained harmful output. |
| Approach: | They propose a method that directly modifies model weights to map a trigger to an attacker-specified response. |
| Outcome: | The proposed method achieves high triggered attack success while maintaining non-triggered safety and general utility. |
Discover and Prove: An Open-source Agentic Framework for Hard Mode Automated Theorem Proving in Lean 4 (2026.acl-long)
Copied to clipboard
Chengwu Liu, Yichun Yin, Ye Yuan, Jiaxuan Xie, Botao Li, Siqi Li, Jianhao Shen, Yan Xu, Lifeng Shang, Ming Zhang
| Challenge: | Existing approaches to solving mathematical problems fall into two broad categories: informal methods and formal methods. |
| Approach: | They propose to use LLM natural-language reasoning to discover answers . they introduce Discover And Prove framework that rewrites Hard Mode statements into Easy Mode ones for existing ATP provers. |
| Outcome: | The proposed framework can be used to prove hard mode statements on ATP benchmarks. |
Ambiguity Awareness Optimization: Towards Semantic Disambiguation for Direct Preference Optimization (2025.emnlp-main)
Copied to clipboard
| Challenge: | Direct Preference Optimization (DPO) is a widely used reinforcement learning from human feedback (RLHF) method across various domains. |
| Approach: | They propose an approach that automatically re-weights ambiguous content to reduce ambiguities by calculating semantic similarity from preference pairs. |
| Outcome: | The proposed approach outperforms state-of-the-art approaches in performance across multiple model scales and widely adopted benchmark datasets. |
Consistency Regularization Training for Compositional Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing neural models have difficulty generalizing to unseen combinations of seen components. |
| Approach: | They propose to improve the capability of Transformer on compositional generalization by consistency regularization training without modifying model architectures. |
| Outcome: | The proposed model performs well on semantic parsing and machine translation benchmarks. |
StructBreak: Structural Cognitive Overload-Induced Safety Failures in MLLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work focused on typographic and pixel-level perturbations, leaving the study of SCO unexplored. |
| Approach: | They propose a framework that exploits MLLMs' diagrammatic reasoning capabilities to bypass safety guardrails. |
| Outcome: | The proposed framework exploits the model's reasoning capabilities to bypass safety guardrails. |
Why Safeguarded Ships Run Aground? Aligned Large Language Models’ Safety Mechanisms Tend to Be Anchored in The Template Region (2025.acl-long)
Copied to clipboard
| Challenge: | Infilling a fixed template between the input instruction and initial model output is a common practice for existing LLMs, but it is vulnerable to inference-time jailbreak attacks. |
| Approach: | They propose to fill a fixed template between the input instruction and initial model output and to detach safety mechanisms from the template region to mitigate the risk of inference-time jailbreak attacks. |
| Outcome: | The proposed method is widespread across aligned LLMs and shows that it mitigates inference-time jailbreak vulnerabilities. |
Proactive Guidance of Multi-Turn Conversation in Industrial Search (2025.acl-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced multi-turn conversation systems, emphasizing the need for proactive guidance to enhance users’ interactions. |
| Approach: | They propose a goal-adaptive supervised fine-tuning framework that generates proactive guidance for users to click for the next turn of the conversation. |
| Outcome: | The proposed framework achieves 86.10% accuracy in offline evaluation (+23.95% over baseline) and 25.28% CTR in online deployment (149.06% relative improvement). |
Generation and Extraction Combined Dialogue State Tracking with Hierarchical Ontology Integration (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current models are not satisfactory for solving out-of-vocabulary problems . current models assume that the task ontology is well defined in advance . |
| Approach: | They propose to enhance the interrelation between slots with masked hierarchical attention. |
| Outcome: | The proposed model yields a significant performance gain over current state-of-the-art model and is more robust to out-ofvocabulary problem compared with other methods. |
AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code (2026.findings-acl)
Copied to clipboard
ShangZhan Li, Xinyu Yin, Xuanyu Jin, Ye He, Yuxin Zhou, Yuxuan Li, Xu Han, Wanxiang Che, Qi Shi, Ting Liu, Maosong Sun
| Challenge: | Current development practices face a dichotomy between automation and performance. |
| Approach: | They propose a framework to empower LLMs with the capability of automated explicit vectorization. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the SSE and AVX subsets of SimdBench. |
IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce (2024.findings-emnlp)
Copied to clipboard
Wenxuan Ding, Weiqi Wang, Sze Kwok, Minghao Liu, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Junxian He, Yangqiu Song
| Challenge: | Existing approaches that distill intentions from LMs fail to generate meaningful and human-centric intentions applicable in real-world E-commerce contexts. |
| Approach: | They propose a double-task multiple-choice question answering benchmark to evaluate LMs' comprehension of purchase intentions in E-commerce. |
| Outcome: | The proposed benchmark consists of 4,360 carefully curated problems across three difficulty levels, constructed using an automated pipeline to ensure scalability on large E-commerce platforms. |
Attribute-aware Sequence Network for Review Summarization (D19-1)
Copied to clipboard
| Challenge: | Existing review summarization systems generate summary only based on review content and neglect the authors’ attributes (e.g., gender, age, and occupation). |
| Approach: | They propose an Attribute-aware Sequence Network (ASN) to take the aforementioned users’ characteristics into account by encoding their attributes over the words. |
| Outcome: | The proposed model outperforms existing systems on tripAtt and human evaluation by taking the authors' attributes into account and incorporating attribute embedding and word-using habits into word prediction. |
Answering Cross-Dimensional Geometric Visual Questions by Multi-constraint Spatial Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for solving complex visual questions are limited in their ability to represent in a cross-dimensional space. |
| Approach: | They propose a method that can answer complex visual questions using cross-dimensional reasoning. |
| Outcome: | The proposed method can answer complex visual questions in 2D to 3D space with great application value. |
TinyBERT: Distilling BERT for Natural Language Understanding (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models are computationally expensive and difficult to efficiently execute on resource-restricted devices. |
| Approach: | They propose a Transformer distillation method that performs Transformer distillations at pre-training and task-specific learning stages. |
| Outcome: | The proposed method accelerates inference and reduces model size while maintaining accuracy. |
A Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing knowledge-grounded dialogues perform poorly when transfer into new domains with limited training samples. |
| Approach: | They propose a weakly supervised three-stage learning framework based on weakly-supervised learning based upon large scale ungrounded dialogues and unstructured knowledge base. |
| Outcome: | The proposed framework outperforms state-of-the-art methods even in zero-resource setting. |
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding (2023.emnlp-main)
Copied to clipboard
| Challenge: | 3D visual grounding aims to localize the desired objects in a 3D point cloud by a free-form language description. |
| Approach: | They propose a relation-aware framework which captures relative spatial relationships between objects and enhances object attributes. |
| Outcome: | The proposed framework outperforms state-of-the-art methods on three benchmarks . it captures relative spatial relationships between objects and enhances object attributes . |
Few-shot Temporal Pruning Accelerates Diffusion Models for Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing acceleration methods for text generation ignore the importance of the distribution of sampling steps, resulting in slow sampling rates. |
| Approach: | They propose a technique to accelerate diffusion models for text generation without additional training by using a Bayesian optimization approach. |
| Outcome: | The proposed technique achieves 400x acceleration even with minimal sampling steps after down to less than 1 minute of optimization yielding a competitive performance even with minimum sampling steps. |
Mixed Distillation Helps Smaller Language Models Reason Better (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated impressive multiple step-by-step reasoning capabilities in recent NLP reasoning tasks. |
| Approach: | They propose a mixed distillation framework that distills multiple step-by-step reasoning abilities into smaller language models (SLMs) they leverage LLMs to generate multiple step by step reasoning rationales by sampling automatically. |
| Outcome: | The proposed framework outperforms existing models on SVAMP, GSM8K and ASDIV, while a single model generated by MD exceeds the comprehensive performance of two individual CoT and PoT distilled models. |
NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation (2023.acl-long)
Copied to clipboard
Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Ming Gong, Lijuan Wang, Zicheng Liu, Houqiang Li, Nan Duan
| Challenge: | Existing work generates long videos segment by segment sequentially, which is inefficient. |
| Approach: | They propose a Diffusion over Difference architecture for eXtremely Long video generation. |
| Outcome: | The proposed architecture reduces the average inference time from 7.55min to 26s (94.26%) and generates high-quality long videos with both global and local coherence. |
Unified Active Retrieval for Retrieval Augmented Generation (2024.findings-emnlp)
Copied to clipboard
Qinyuan Cheng, Xiaonan Li, Shimin Li, Qin Zhu, Zhangyue Yin, Yunfan Shao, Linyang Li, Tianxiang Sun, Hang Yan, Xipeng Qiu
| Challenge: | Existing active retrieval methods struggle with handling various types of instructions. |
| Approach: | They propose a unified active retrieval framework for retrieval-augmented generation . they propose to combine four orthogonal criteria into plug-and-play classification tasks . |
| Outcome: | The proposed framework outperforms existing methods on four representative types of user instructions on four types of instructions. |
LEDOM: Reverse Language Model (2026.acl-long)
Copied to clipboard
Xunjian Yin, Sitao Cheng, Yuxi Xie, Xinyu Hu, Li Lin, Xinyi Wang, Liangming Pan, William Yang Wang, Xiaojun Wan
| Challenge: | Autoregressive language models are trained exclusively left-to-right, yet they are limited in their ability to factorize text. |
| Approach: | They propose a purely reverse autoregressive language model that factorizes text as a product of left-to-right conditionals. |
| Outcome: | The proposed model can be used to score forward outputs using reverse posterior estimates. |
"I Don’t Know What to Say": A Fact-Filling Questionnaire Method to Help Non-Experts Talk to LegalAI Assistant (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have greatly expanded the scope of legal AI. |
| Approach: | They propose a method that generates questionnaires to help users refine queries . they leverage an iterative training process that collects valuable questionnaires . |
| Outcome: | The proposed method improves the completeness of queries and ensures the performance of domain-specific models in downstream legal tasks. |
BrowseConf: Confidence-Guided Test-Time Scaling for Web Agents (2026.findings-acl)
Copied to clipboard
Litu Ou, Kuan Li, Huifeng Yin, Liwen Zhang, Zhongwang Zhang, Xixi Wu, Rui Ye, Zile Qiao, Yong Jiang, Pengjun Xie, Fei Huang, Jingren Zhou
| Challenge: | Existing work on confidence in LLMs is limited. |
| Approach: | They propose to use confidence scores to determine model answer quality and encourage model to try again until it reaches satisfactory confidence level. |
| Outcome: | The proposed methods significantly reduce token consumption while demonstrating competitive performance compared to baseline fixed budget methods. |
Entity-Relation Extraction as Multi-Turn Question Answering (P19-1)
Copied to clipboard
| Challenge: | Identifying entities and their relations is the prerequisite of extracting structured knowledge from unstructured raw texts. |
| Approach: | They propose a new paradigm for the task of entity-relation extraction . they cast the task as a multi-turn question answering problem . |
| Outcome: | The proposed paradigm significantly outperforms previous best models on the ACE and CoNLL04 datasets. |
from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have exposed the risk of Large Language Models (LLMs) generating harmful content by jailbreak attacks. |
| Approach: | They propose a framework that exploits AdVersArial meTAphoR to induce LLMs to calibrate harmful metaphors for jailbreaking. |
| Outcome: | The proposed framework can successfully jailbreak Large Language Models (LLMs) by leveraging the AdVersArial meTAphoR (AVATAR) framework achieves state-of-the-art attack success rate across multiple advanced LLMs. |
Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have explored using large language models to generate synthetic datasets . however, the effectiveness of the LLM-generated synthetic data is inconsistent across different classification tasks. |
| Approach: | They propose to use large language models to generate synthetic datasets to better understand factors that moderate the effectiveness of LLM-generated synthetic data. |
| Outcome: | The results show that subjectivity is negatively associated with the performance of the model trained on synthetic data. |
OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use (2025.acl-long)
Copied to clipboard
Xueyu Hu, Tao Xiong, Biao Yi, Zishu Wei, Ruixuan Xiao, Yurun Chen, Jiasheng Ye, Meiling Tao, Xiangxin Zhou, Ziyu Zhao, Yuhuai Li, Shengze Xu, Shenzhi Wang, Xinchen Xu, Shuofei Qiao, Zhaokai Wang, Kun Kuang, Tieyong Zeng, Liang Wang, Jiwei Li, Yuchen Eleanor Jiang, Wangchunshu Zhou, Guoyin Wang, Keting Yin, Zhou Zhao, Hongxia Yang, Fan Wu, Shengyu Zhang, Fei Wu
| Challenge: | a new generation of (M)LLMs is enabling the creation of superintelligent AI assistants . OS Agents can complete tasks autonomously and have the potential to significantly enhance the lives of billions of users worldwide. |
| Approach: | They propose to build OS Agents that operate within operating systems' GUIs and GUIs . they examine evaluation metrics and benchmarks to identify promising directions . |
| Outcome: | The proposed agents are based on operating systems (OS) and operating systems frameworks. |
SCOTT: Self-Consistent Chain-of-Thought Distillation (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LMs) generate free-text rationales for their predictions via chain-of-thought prompting, but there is little guarantee that the generated rationale is consistent with LM’s predictions or faithfully justify the decisions. |
| Approach: | They propose a faithful knowledge distillation method to learn a small, self-consistent CoT model from a larger teacher model by contrastive decoding. |
| Outcome: | The proposed method yields comparable performance but is less faithful than baselines. |
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation (2023.acl-long)
Copied to clipboard
Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao
| Challenge: | Existing models for speech-to-speech translation suffer from distinct degradation in noisy environments and fail to translate visual speech. |
| Approach: | They propose a text-based audio-visual speech-to-speech translation model that integrates visual information with audio-only data to improve system robustness. |
| Outcome: | The proposed model outperforms models trained on audio-only corpus in two languages . it also improves with low-resource audio-visual data, compared with baselines . |
Continual Few-shot Intent Detection (2022.coling-1)
Copied to clipboard
| Challenge: | Existing intent detection systems are trained with lots of labeled data over a predefined set of intent classes. |
| Approach: | They propose a prefix-guided lightweight encoder with three auxiliary strategies to prevent catastrophic forgetting and negative knowledge transfer across tasks. |
| Outcome: | The proposed system prevents catastrophic forgetting and encourages positive knowledge transfer across tasks. |
Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning (2026.acl-long)
Copied to clipboard
Zelin Tan, Hejia Geng, Xiaohang Yu, Mulei Zhang, Guancheng Wan, Yifan Zhou, Qiang He, Xiangyuan Xue, Heng Zhou, Yutao Fan, Zhong-Zhi Li, Zaibin Zhang, Guibin Zhang, Chen Zhang, Zhenfei Yin, Philip Torr, Lei Bai
| Challenge: | elucidating scaling laws for large language models (LLMs) during pre-training remains unexplored. |
| Approach: | They characterize how model scale, data, and compute interact during pre-training . they find that large models consistently demonstrate superior compute and data efficiency . |
| Outcome: | The proposed scaling laws offer practical guidance for scaling reasoning capabilities through reinforcement learning post-training. |
CBLUE: A Chinese Biomedical Language Understanding Evaluation Benchmark (2022.acl-long)
Copied to clipboard
Ningyu Zhang, Mosha Chen, Zhen Bi, Xiaozhuan Liang, Lei Li, Xin Shang, Kangping Yin, Chuanqi Tan, Jian Xu, Fei Huang, Luo Si, Yuan Ni, Guotong Xie, Zhifang Sui, Baobao Chang, Hui Zong, Zheng Yuan, Linfeng Li, Jun Yan, Hongying Zan, Kunli Zhang, Buzhou Tang, Qingcai Chen
| Challenge: | a new benchmark for biomedical language understanding is being developed in Chinese . most benchmarks are limited to English, which makes it difficult to replicate success in other languages. |
| Approach: | They propose to use Chinese biomedical language understanding evaluation benchmarks to evaluate Chinese models. |
| Outcome: | The proposed benchmarks show that the current models perform worse than the human ceiling. |
NewsDialogues: Towards Proactive News Grounded Conversation (2023.findings-acl)
Copied to clipboard
Siheng Li, Yichun Yin, Cheng Yang, Wangjie Jiang, Yiwei Li, Zesen Cheng, Lifeng Shang, Xin Jiang, Qun Liu, Yujiu Yang
| Challenge: | Hot news is one of the most popular topics in daily conversations. |
| Approach: | They propose a task where a dialogue system can lead the conversation based on key topics of the news. |
| Outcome: | The proposed method can lead conversations based on key topics of the news . it can also be used in information-seeking and chit-chat scenarios . |
How Does the Disclosure of AI Assistance Affect the Perceptions of Writing? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in generative AI technologies like large language models have boosted the incorporation of AI assistance in writing workflows. |
| Approach: | They conduct an experimental study to determine whether disclosure of AI assistance in the writing process would affect people's evaluation on the quality of the writing and ranking of different writings. |
| Outcome: | The disclosure of AI assistance decreases the average quality ratings for argumentative essays and creative stories, and increases the quality of the writings. |
Behavioral Consistency Validation for LLM Agents: An Analysis of Trading-Style Switching through Stock-Market Simulation (2026.findings-acl)
Copied to clipboard
Zeping Li, Guancheng Wan, Keyang Chen, Yu Chen, Yiwen Zhao, Philip Torr, Guangnan Ye, Zhenfei Yin, Hongfeng Chai
| Challenge: | Recent studies have applied Large Language Models (LLMs) as agents in financial stock market simulations to test if micro-level behaviors aggregate into macro-level phenomena. |
| Approach: | They propose four alignment metrics and use Mann–Whitney U tests to compare agents’ style-switching behavior with financial theory. |
| Outcome: | The proposed model is only partially consistent with financial theory. |
AutoUE: Automated Generation of 3D Games in Unreal Engine via Multi-Agent Systems (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) and generative models have motivated studies on automated game generation from natural language descriptions. |
| Approach: | They propose a novel multi-agent system, AutoUE, which coordinates multiple agents to end-to-end generate 3D games, covering model retrieval, scene generation, gameplay and interaction code synthesis, and automated game testing for evaluation. |
| Outcome: | The proposed system covers model retrieval, scene generation, gameplay and interaction code synthesis, and automated game testing for evaluation. |
Rethinking the Construction of Effective Metrics for Understanding the Mechanisms of Pretrained Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained language models are expected to map input text to a set of vectors while preserving the inherent relationships within the text. |
| Approach: | They propose a new line to constructing metrics for understanding the mechanisms of pretrained language models. |
| Outcome: | The proposed model is referred to as the tree topological probe and is based on a tree topology probe. |
SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding (2026.findings-acl)
Copied to clipboard
Yuqi Yang, Weiqi Wang, Baixuan Xu, Wei Fan, Qing Zong, Chunkit Chan, Zheye Deng, Xin Liu, Yifan Gao, Changlong Yu, Chen Luo, Yang Li, Zheng Li, Qingyu Yin, Bing Yin, Yangqiu Song
| Challenge: | Existing models fail to capture and model customer intention effectively because of insufficient information exploitation and only apparent information like descriptions and titles are used. |
| Approach: | They propose to exploit existing session data to capture and model intention in E-commerce product purchase sessions using a multimodal benchmark. |
| Outcome: | The proposed framework can bridge the gap between intention understanding in simplified research cases like co-buy intention and more complex yet practical scenarios like session history. |
FolkScope: Intention Knowledge Graph Construction for E-commerce Commonsense Discovery (2023.findings-acl)
Copied to clipboard
Changlong Yu, Weiqi Wang, Xin Liu, Jiaxin Bai, Yangqiu Song, Zheng Li, Yifan Gao, Tianyu Cao, Bing Yin
| Challenge: | Existing intention-based studies on recommendation tasks are limited and use models to implicitly model the intention memberships. |
| Approach: | They propose a framework that leverages the generation power of large language models and human-in-the-loop annotation to semi-automatically construct the intention knowledge graph. |
| Outcome: | The proposed framework can model e-commerce knowledge and have many potential applications. |
AutoTaskEval: Towards Domain-Specific and Fine-Grained Evaluation for LLMs (2026.acl-long)
Copied to clipboard
Qingqing Lyu, Linjuan Wu, Yongliang Shen, Hengwei Liu, Hao Li, Shengpei Jiang, Yin Zhang, Weiming Lu
| Challenge: | Existing automated approaches operate within fixed task schemas and often fail to autonomously discover new evaluation dimensions. |
| Approach: | They propose an automated framework that constructs domain-specific benchmarks directly from unstructured corpora using Bloom’s Taxonomy. |
| Outcome: | The proposed framework uncovers a broader and more fine-grained task space than expert-curated benchmarks while producing high-quality instances that preserve established model-level evaluation trends. |
Outlier-weighed Layerwise Sampling for LLM Fine-tuning (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a powerful tool for processing complex natural language processing tasks. |
| Approach: | They propose an approach to fine-tune LLMs with outliers and a gradient low-rank projection to increase the number of fine-sampled layers without a proportional increase in memory costs. |
| Outcome: | The proposed approach outperforms baseline approaches while being more memory efficient. |
On-Policy Self-Distillation for Efficient Diffusion Language Models with Early-Stage Calibration (2026.findings-acl)
Copied to clipboard
Huaisheng Zhu, MingYu Liu, Junze Liu, Zhen Ge, Tian Wang, Jiri Gesi, Dakuo Wang, Weiqi Zhang, Houyu Zhang, Yufan Guo, Xian Li, Bing Yin, Sujay Sanghavi
| Challenge: | Recent studies have demonstrated that masked diffusion models (MDMs) can surpass autoregressive models (ARMs) in various tasks. |
| Approach: | They propose a method to calibrate early token predictions without demonstration data by distilling an unnormalized target distribution into the original model. |
| Outcome: | Experiments on math, planning, and RLHF tasks show that COPSD improves both effectiveness and efficiency, and further enhances performance when combined with supervised fine-tuning. |
Answering Complex Geographic Questions by Adaptive Reasoning with Visual Context and External Commonsense Knowledge (2025.acl-long)
Copied to clipboard
| Challenge: | a new task of answering geographic reasoning questions based on the given image is proposed . the task requires identifying the objects in the image and understanding the background context . |
| Approach: | They propose a task of answering geographic reasoning questions based on the given image . they analyze the image and describe its fine-grained content by text and keywords . |
| Outcome: | The proposed method can be used to answer geographic reasoning questions based on an image . it can be applied to a large-scale dataset with 41,329 samples . |
Prompt-Driven Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation models still face various challenges including fragility and lack of style flexibility. |
| Approach: | They propose to incorporate prompts into neural machine translation to improve translation control and style flexibility. |
| Outcome: | Empirical results show that the proposed method improves translation control and quality and improves human-in-the-loop translation. |
Quantized but Deceptive? A Multi-Dimensional Truthfulness Evaluation of Quantized LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Quantization enables efficient deployment of large language models in resource-constrained environments . but impact on truthfulness remains largely unexplored . |
| Approach: | They propose a framework to assess the truthfulness of quantized large language models . they find quantized models retain internally truthful representations but produce false outputs . |
| Outcome: | The framework assesses the truthfulness of quantized models across three dimensions . it finds that quantized model models retain internally truthful representations but are more susceptible to false outputs . |
What Does BERT with Vision Look At? (2020.acl-main)
Copied to clipboard
| Challenge: | Pre-trained visual grounded language models have improved performance on vision-and-language tasks but what they learn during pre-training remains unclear. |
| Approach: | They show that attention heads of visual grounded language models actively ground elements of language to image regions. |
| Outcome: | The attention heads of a visual grounded language model can ground elements to image regions, demonstrating their ability to detect syntactic relations between non-entity words and image regions. |
Aggregation of Reasoning: A Hierarchical Framework for Enhancing Answer Selection in Large Language Models (2024.lrec-main)
Copied to clipboard
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Zhiyuan Zeng, Xiaonan Li, Tianxiang Sun, Cheng Chang, Qinyuan Cheng, Ding Wang, Xiaofeng Mou, Xipeng Qiu, Xuanjing Huang
| Challenge: | Recent advances in Chain-of-Thought prompting have facilitated significant breakthroughs for Large Language Models (LLMs) in complex reasoning tasks. |
| Approach: | They propose a hierarchical reasoning aggregation framework to address this problem . they propose dynamic sampling to adjust the number of reasoning chains . |
| Outcome: | The proposed framework outperforms existing ensemble methods on complex reasoning tasks. |
MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL (2025.coling-main)
Copied to clipboard
Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, LinZheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, Zhoujun Li
| Challenge: | Recent LLM-based Text-to-SQL methods suffer from performance degradation on “huge” databases and complex user questions that require multi-step reasoning. |
| Approach: | They propose a framework that integrates a decomposer agent and auxiliary agents to generate SQL queries from natural language text. |
| Outcome: | The proposed framework achieves comparable execution accuracy on SQL-Llama tasks compared to the baseline model. |
Learn to Cross-lingual Transfer with Meta Graph Learning Across Heterogeneous Languages (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing mPLM-based methods focus on designing costly model pre-training while ignoring equally crucial downstream adaptation. |
| Approach: | They propose a meta graph learning method that extracts meta-knowledge from historical CLT experiences to learn to cross-lingual transfer. |
| Outcome: | The proposed method can learn to cross-lingual transfer by extracting meta-knowledge from historical CLT experiences (tasks) it can also capture intrinsic language relationships to explicitly guide cross-linguistic transfer. |
Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies on keyphrase generation on non-English languages haven’t been vastly investigated. |
| Approach: | They propose a retrieval-augmented method for multilingual keyphrase generation that leverages keyphrase annotations in English datasets to facilitate generating keyphrases in low-resource languages. |
| Outcome: | The proposed model outperforms baselines on non-English keyphrase generation datasets and the proposed model is scalable. |
ContextCheck: Sentence-Level Faithfulness Verification with Context-Aware Disambiguation (2026.findings-acl)
Copied to clipboard
Yueqin Yin, Yaxi Li, Xin Liu, Xun Wang, Kaiqiang Song, Simin Ma, Shujian Liu, Sathish Reddy Indurthi, Haoyun Deng, Pengcheng He, Mingyuan Zhou, Song Wang
| Challenge: | Large language models often hallucinate, producing content that is factually incorrect or not grounded in the sources. |
| Approach: | They propose a framework for sentence-level faithfulness verification with context-aware disambiguation. |
| Outcome: | The proposed framework improves Macro F1 by over 10 points compared to baselines on three context-dependent datasets. |
Trial and Error: Exploration-Based Trajectory Optimization of LLM Agents (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have become integral components in various autonomous agent systems. |
| Approach: | They propose an exploration-based trajectory optimization approach that allows agents to learn from their exploration failures. |
| Outcome: | The proposed method outperforms baseline methods on three complex tasks by a large margin. |