Papers by Qi Yu
Verified Critical Step Optimization for LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Critical Step Optimization (CSO) focuses preference learning on verified critical steps where alternative actions demonstrably flip task outcomes from failure to success. |
| Approach: | They propose a method which focuses preference learning on verified critical steps where alternative actions demonstrably flip task outcomes from failure to success. |
| Outcome: | The proposed method outperforms the existing methods on GAIA-Text-103 and XBench-DeepSearch while requiring supervision at only 16% of trajectory steps. |
Through the MUD: A Multi-Defendant Charge Prediction Benchmark with Linked Crime Elements (2024.acl-long)
Copied to clipboard
| Challenge: | Existing charge prediction datasets focus on single-defendant cases, but real-world cases involve multiple defendants. |
| Approach: | They propose a benchmark that encompasses legal cases involving multiple defendants . they develop an interpretable model called EJudge that incorporates crime elements and legal rules to infer charges. |
| Outcome: | The proposed model outperforms state-of-the-art models in predicting crime charges while providing corresponding rationales. |
Augmenting Multi-Turn Text-to-SQL Datasets with Self-Play (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Numerous architectures and pretraining methods have been proposed for context-dependent text-to-SQL, but the size of the datasets used has been limited due to the high cost of annotating multi-turn dialogue and SQL pairs. |
| Approach: | They propose to augment training datasets using self-play which leverages contextual information to synthesize new interactions to adapt the model to new databases. |
| Outcome: | The proposed model improves accuracy on SParC and CoSQL, two widely used cross-domain text-to-SQl datasets. |
Patton: Language Model Pretraining on Text-Rich Networks (2023.acl-long)
Copied to clipboard
| Challenge: | Existing models for text-rich networks do not take inter-document structure into account. |
| Approach: | They propose a pretraining framework for a text-rich network using a masked language model and a masking node prediction framework. |
| Outcome: | The proposed model outperforms baselines on four tasks in academic and e-commerce domains. |
Rethinking LLM Uncertainty: A Multi-Agent Approach to Estimating Black-Box Model Uncertainty (2025.findings-emnlp)
Copied to clipboard
Yu Feng, Phu Mon Htut, Zheng Qi, Wei Xiao, Manuel Mager, Nikolaos Pappas, Kishaloy Halder, Yang Li, Yassine Benajiba, Dan Roth
| Challenge: | Existing methods to gauge model’s uncertainty through self-consistency in responses to the target query are misleading: an LLM may confidently provide an incorrect answer to a target query, yet give a confident and accurate answer to that same query when answering a knowledge-preserving perturbation of the query. |
| Approach: | They propose a method that uses multi-agent interaction to estimate black-box LLMs' uncertainty. |
| Outcome: | The proposed method outperforms existing self-consistency based methods and improves hallucination detection. |
DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections (2026.findings-acl)
Copied to clipboard
| Challenge: | Several studies rely on additional models to optimize mixtures. |
| Approach: | They propose a method that dynamically optimizes instruction-tuning dataset mixtures by prior-scaled Boltzmann Exploration and a multi-armed bandit setup. |
| Outcome: | The proposed method improves the TÜLU-2-mixture and TÜLO-3-mixtures across 10 benchmarks while introducing minimal computational overhead over naive sampling. |
CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios (2025.emnlp-main)
Copied to clipboard
| Challenge: | a number of tools are used to perform complex tasks, but the tool utilization process can cause errors. |
| Approach: | They propose a critique evaluation benchmark for tool learning that analyzes function-calling errors on tool evaluation benchmarks. |
| Outcome: | The proposed critique evaluation benchmark holds diverse tool-use errors with varying complexities, which better reflects real-world scenarios. |
Multilingual BERT Post-Pretraining Alignment (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work improves on the success of monolingual pretrained language models by adding cross-lingual tasks that always involve English. |
| Approach: | They propose a method to align multilingual contextual embeddings as a post-pretraining step for improved cross-lingual transferability of pretrained language models. |
| Outcome: | The proposed model outperforms XLM-R_Base on translation-train tasks while using less parallel data and fewer parameters. |
FOLIO: Natural Language Reasoning with First-Order Logic (2024.emnlp-main)
Copied to clipboard
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, Lucy Sun, Alexander Wardle-Solano, Hannah Szabó, Ekaterina Zubova, Matthew Burtell, Jonathan Fan, Yixin Liu, Brian Wong, Malcolm Sailor, Ansong Ni, Linyong Nan, Jungo Kasai, Tao Yu, Rui Zhang, Alexander Fabbri, Wojciech Kryscinski, Semih Yavuz, Ye Liu, Xi Lin, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Rex Ying, Arman Cohan, Dragomir Radev
| Challenge: | Existing benchmarks for logical reasoning in large language models lack language naturalness or limited complexity. |
| Approach: | They propose to use first-order logic annotations to evaluate logical reasoning capabilities of large language models. |
| Outcome: | The proposed dataset evaluates the FOL reasoning ability of supervised fine-tuning on medium-sized language models. |
Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to augment pre-trained large language models require extensive computational efforts and massive data volumes, challenging the widespread accessibility of LLM research. |
| Approach: | They propose a post-pretraining strategy of selectively enhancing shallow layers while pruning less effective deep ones to augment pretrained large language models. |
| Outcome: | The proposed approach improves performance on the corpus of code & math and a legal corpus and is widely applicable. |
PepRec: Progressive Enhancement of Prompting for Recommendation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been gaining in-depth performance in natural language processing domains. |
| Approach: | They propose a training-free prompting framework that captures knowledge from content-based filtering and collaborative filtering to boost recommendation performance with LLMs. |
| Outcome: | The proposed framework outperforms traditional deep learning recommendation models and prompt-based recommendation systems on two real-world datasets. |
Text Fluoroscopy: Detecting LLM-Generated Text through Intrinsic Features (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized the field of natural language processing because of their excellent performance on various tasks. |
| Approach: | They propose a black-box method with better generalizability for detecting LLM-generated text by mining the intrinsic features of the text to be detected. |
| Outcome: | The proposed method achieves 7.36% and 2.84% improvement in detection performance compared to baselines in detecting texts from different domains generated by GPT-4 and Claude3 respectively. |
Figurative Language Processing: A Linguistically Informed Feature Analysis of the Behavior of Language Models and Humans (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent years have witnessed a growing interest in investigating what Transformer-based language models (TLMs) actually learn from training data. |
| Approach: | They propose to use a black-box TLM and two intrinsically transparent white-box models to investigate the performance of figurative language models on sarcasm, similes, idioms, and metaphors. |
| Outcome: | The proposed models perform better than other models on figurative language classification tasks. |
I-AM-G: Interest Augmented Multimodal Generator for Item Personalization (2024.emnlp-main)
Copied to clipboard
| Challenge: | e-commerce and recommender systems lack a framework for personalized generation . a new framework extracts tags from multimodal information of items that the user has interacted with . |
| Approach: | They propose a framework that extracts tags from multimodal information and rewrites item description . they then use a decoupled text-to-text and image-to image retriever to search for similar item text . |
| Outcome: | The proposed framework can generate results aligned with user preferences . it can be used in e-commerce and recommender systems to win over diverse user base . |
Benchmarking Commercial Intent Detection Services with Practice-Driven Evaluations (2021.naacl-industry)
Copied to clipboard
| Challenge: | Intent detection models require large amounts of labeled data to achieve high accuracy, and in practical scenarios it is more common to find small, unbalanced, and noisy datasets. |
| Approach: | They benchmark intent detection methods on a variety of datasets and found that Watson Assistant's model outperforms other commercial solutions. |
| Outcome: | The proposed model outperforms pretrained language models on a variety of datasets while requiring only a fraction of computational resources and training data. |
Bit-Flip Error Resilience in LLMs: A Comprehensive Analysis and Defense Framework (2025.emnlp-main)
Copied to clipboard
Yuhang Chen, Zhen Tan, Ajay Kumar Jaiswal, Huaizhi Qu, Xinyu Zhao, Qi Lin, Yu Cheng, Andrew Kwong, Zhichao Cao, Tianlong Chen
| Challenge: | Bit-flip errors (BFEs) are hardware faults where individual bits in memory or processing units are unintentionally flipped. |
| Approach: | They propose a novel defense strategy to mitigate bit-flip errors (BFEs) they propose bfe protection and a self-correction mechanism to minimize performance degradation . |
| Outcome: | The proposed defense strategy minimizes performance degradation while significantly improving robustness against BFEs. |
UFO: A UI-Focused Agent for Windows OS Interaction (2025.naacl-long)
Copied to clipboard
Chaoyun Zhang, Liqun Li, Shilin He, Xu Zhang, Bo Qiao, Si Qin, Minghua Ma, Yu Kang, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Zhang
| Challenge: | UFO is a UI-Fcused agent designed to fulfill user requests tailored to Windows OS applications . it decomposes user requests using divide-and-conquer approach, enabling seamless navigation and addressing sub-tasks across multiple applications. |
| Approach: | They propose a UI-Fcused Windows OS agent that decomposes user requests using a divide-and-conquer approach and incorporates a control interaction module tailored for Windows OS. |
| Outcome: | The proposed agent decomposes user requests using divide-and-conquer approach, enabling seamless navigation and addressing sub-tasks across multiple applications. |
Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models (2026.findings-acl)
Copied to clipboard
Hengyuan Zhang, Zhihao Zhang, Ercong Nie, Mingyang Wang, Zunhai Su, Yiwei Wang, Qianli Wang, Shuzhou Yuan, Xufeng Duan, Qibo Xue, Zeping Yu, Chenming Shang, Xiao Liang, Jing Xiong, Hui Shen, Chaofan Tao, Zhengwu Liu, Senjie Jin, Zhiheng Xi, Dongdong Zhang, Sophia Ananiadou, Tao Gui, Ruobing Xie, Hayden Kwok-Hay So, Hinrich Schuetze, Xuanjing Huang, Qi Zhang, Ngai Wong
| Challenge: | Existing literature on mechanistic interpretation (MI) treats it as an observational science, leaving practical applications underexplored. |
| Approach: | They propose a survey structured around the pipeline to identify and improve MI models. |
| Outcome: | The proposed framework enables tangible improvements in Alignment, Capability, and Efficiency. |
DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scale (2025.findings-acl)
Copied to clipboard
Linghao Zhang, Junhao Wang, Shilin He, Chaoyun Zhang, Yu Kang, Bowen Li, Jiaheng Wen, Chengxing Xie, Maoquan Wang, Yufan Huang, Elsie Nallipogu, Qingwei Lin, Yingnong Dang, Saravan Rajmohan, Dongmei Zhang, Qi Zhang
| Challenge: | Existing studies highlight that dependency-related issues cause over 40% of observed runtime errors on the generated repository. |
| Approach: | They propose a large-scale benchmark and evaluation framework specifically designed to assess LLMs’ capability on dependency inference. |
| Outcome: | The proposed model achieves only a 48% execution pass rate on Python, indicating room for improvement. |
Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers (2026.findings-acl)
Copied to clipboard
Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan, Naoto Yokoya
| Challenge: | a new study examines the performance of code-switching IR in monolingual contexts . code-witching is a pervasive linguistic phenomenon in global communication . |
| Approach: | They propose a benchmark to evaluate code-switching IR in monolingual contexts . they propose CS-MTEB, which measures performance declines of up to 27% . |
| Outcome: | The proposed benchmark shows that code-switching performance is degraded by 27% . the proposed benchmark is based on a dataset of mixed-language queries . |
Granular Entity Mapper: Advancing Fine-grained Multimodal Named Entity Recognition and Grounding (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for fine-grained content extraction are limited by long-tailed distribution of textual entity categories and performance of object detectors. |
| Approach: | They propose a multi-granularity entity recognition module and a reranking module to integrate hierarchical information of entity categories, visual cues, and external textual resources collectively. |
| Outcome: | The proposed framework achieves state-of-the-art on the fine-grained content extraction task. |
SOTOPIA-π: Interactive Learning of Socially Intelligent Language Agents (2024.acl-long)
Copied to clipboard
Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Yonatan Bisk, Graham Neubig, Hao Zhu
| Challenge: | Existing studies on building language agents have not addressed this social learning gap. |
| Approach: | They propose an interactive learning method that improves the social intelligence of language agents by using behavior cloning and self-reinforcement based training on filtered social interaction data. |
| Outcome: | The proposed method allows a 7B LLM to reach the social goal completion ability of an expert model (GPT-4-based agent) without the loss of more generic abilities, such as the ability to answer knowledge-based questions. |
ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis (2023.findings-acl)
Copied to clipboard
Yakun Yu, Mingjun Zhao, Shi-ang Qi, Feiran Sun, Baoxun Wang, Weidong Guo, Xiaoli Wang, Lei Yang, Di Niu
| Challenge: | Existing methods for multimodal sentiment analysis focus on general knowledge, which is inadequate to identify specific sentiments across modalities. |
| Approach: | They propose a method where specific-knowledge representations for each modality can be learned together with general knowledge representations via knowledge injection based on an adapter architecture. |
| Outcome: | The proposed method outperforms all prior methods on three popular benchmarks on multimodal sentiment analysis metrics. |
Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluation benchmarks focus on pairwise matching, ignoring robustness . current models exhibit frustrating degradation, with a maximum drop of 23.43 F1 score . |
| Approach: | They propose a benchmark that simulates the evaluation of open information extraction models in the real world . they perform experiments on typical models published in the last decade and a representative large language model . |
| Outcome: | The proposed model is rated robust on a knowledge-invariant clique with different syntactic and expressive forms. |
CLEAR: A Framework Enabling Large Language Models to Discern Confusing Legal Paragraphs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work focuses on enabling LLMs to leverage legal rules to tackle complex legal reasoning tasks, but ignores their ability to understand legal rules. |
| Approach: | They propose a legal paragraph prediction task that aims to predict the legal paragraph given criminal facts and a framework CLEAR to enhance their legal reasoning ability. |
| Outcome: | The proposed model improves the ability of LLMs to analyze legal cases with the guidance of legal rule insights. |
Self-Error-Instruct: Generalizing from Errors for LLMs Mathematical Reasoning (2025.acl-long)
Copied to clipboard
Erxin Yu, Jing Li, Ming Liao, Qi Zhu, Boyang Xue, Minghui Xu, Baojun Wang, Lanqing Hong, Fei Mi, Lifeng Shang
| Challenge: | Existing approaches to learning from errors synthesize training data by extrapolating from isolated bad cases, thereby failing to generalize the extensive patterns inherent within these cases. |
| Approach: | They propose a framework that synthesizes more generalized training data from isolated bad cases by extrapolating from isolated cases. |
| Outcome: | The proposed framework synthesizes more generalized training data to address these model weaknesses. |
Tiny-NewsRec: Effective and Efficient PLM-based News Recommendation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work fine tunes the PLM with the news recommendation task, which can cause a domain shift problem. |
| Approach: | They propose a self-supervised method to adapt general PLM to news domain with a contrastive matching task between news titles and news bodies. |
| Outcome: | The proposed method can improve both the effectiveness and efficiency of the large PLM-based news recommendation model while maintaining its performance. |
PQR: Improving Dense Retrieval via Potential Query Modeling (2025.acl-long)
Copied to clipboard
| Challenge: | Existing training data is sparse, with each document associated with one or a few labeled queries. |
| Approach: | They propose a training-free potential query retrieval framework to address this problem . they use a Gaussian mixture distribution to model all potential queries for a document . |
| Outcome: | The proposed method is able to capture comprehensive semantic information from a document with multiple queries. |
P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains (2024.findings-emnlp)
Copied to clipboard
Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan
| Challenge: | Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales. |
| Approach: | They propose to annotate a human-annotated dataset consisting of diverse and complex reasoning chains for a set of realistic logical reasoning stories also written by humans. |
| Outcome: | The proposed model outperforms existing methods on understanding the capabilities of LLMs in logical reasoning by 10% or more. |
ResoFilter: Fine-grained Synthetic Data Filtering for Large Language Models through Data-Parameter Resonance Analysis (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for enhancing large language models lack clear metrics for evaluating data characteristics. |
| Approach: | They propose a method that integrates models, data, and tasks to refine datasets. |
| Outcome: | The proposed method achieves comparable results to full-scale fine-tuning using only half the data in mathematical tasks and exhibits strong generalization across different models and domains. |
Capturing the Content of a Document through Complex Event Identification (2022.starsem-1)
Copied to clipboard
| Challenge: | Recent work grouped granular events into more general events, called complex events . however, this approach assumes that a given complex event is always described in consecutive sentences . |
| Approach: | They propose a context-augmented representation learning approach that uses contextual information to model pairwise relation between granular events. |
| Outcome: | The proposed approach outperforms baselines on the complex event identification task. |
TinyScientist: An Interactive, Extensible, and Controllable Framework for Building Research Agents (2025.emnlp-demos)
Copied to clipboard
Haofei Yu, Keyang Xuan, Fenghai Li, Kunlun Zhu, Zijie Lei, Jiaxun Zhang, Ziheng Qi, Kyle Richardson, Jiaxuan You
| Challenge: | Existing research systems often design and use agentic workflows to perform research tasks such as ideation, scientific coding, review writing, and tree-based search. |
| Approach: | They propose an open-source codebase, an interactive web demonstration, and a PyPI Python package to make state-of-the-art auto-research pipelines broadly accessible to every researcher and developer. |
| Outcome: | The proposed framework adapts easily to new tools and supports iterative growth. |
“Again, Dozens of Refugees Drowned”: A Computational Study of Political Framing Evoked by Presuppositions (2022.naacl-srw)
Copied to clipboard
| Challenge: | Earlier studies on issue framing have focused heavily on shallow classification of issue framers, while framerical cues remain neglected. |
| Approach: | They take presupposition-triggering adverbs such as ‘again’ as a study case and examine how different German newspapers use them to covertly evoke different attitudinal subtexts. |
| Outcome: | The findings show that iterative adverbs like 'again' can act as subtle but effective cues of framing. |
SRF: Enhancing Document-Level Relation Extraction with a Novel Secondary Reasoning Framework (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document-level relation extraction ignore bidirectional mention interaction when generating relational features for entity pairs. |
| Approach: | They propose a document-level relation extraction model that incorporates bidirectional mention fusion and a simple yet effective evidence extraction module for relation prediction. |
| Outcome: | The proposed model achieves SOTA performance and the proposed method is effective and general when integrated into existing models. |
DuReadervis: A Chinese Dataset for Open-domain Document Visual Question Answering (2022.findings-acl)
Copied to clipboard
| Challenge: | Open-domain question answering is a task that requires answering questions based on a collection of document images. |
| Approach: | They propose to use document images to answer questions using layouts and visual features instead of text. |
| Outcome: | The proposed approach reduces human cost and improves scalability of QA systems by incorporating layouts and visual features. |
ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sequence-to-sequence models are optimized for future n-gram prediction and n stream self-attention mechanism. |
| Approach: | They propose a self-supervised objective called future n-gram prediction and the proposed n stream self-attention mechanism to optimize the model for sequence-to-sequence learning. |
| Outcome: | The proposed model achieves state-of-the-art on CNN/DailyMail, Gigaword, and SQuAD 1.1 benchmarks compared to the models using the same scale pre-training corpus. |
SpecAgent: A Speculative Retrieval and Forecasting Agent for Code Completion (2026.acl-long)
Copied to clipboard
George Ma, Anurag Koul, Qi Chen, Yawen Wu, Sachit Kuhar, Yu Yu, Aritra Sengupta, Varun Kumar, Murali Krishna Ramanathan
| Challenge: | Large Language Models (LLMs) excel at code-related tasks but struggle in real software repositories. |
| Approach: | They propose a large-scale agent that injects repository context at inference time to improve both latency and code-generation quality by proactively exploring repository files during indexing and constructing speculative context. |
| Outcome: | Experiments show that SpecAgent achieves 9–11% relative performance gains compared to baselines while significantly reducing inference latency. |
Parallelism and Generation Order in Masked Diffusion Language Models: Limits Today, Potential Tomorrow (2026.findings-acl)
Copied to clipboard
Yangyang Zhong, Yanmei Gu, Zhengqing Zang, Xiaomeng Li, Yuqi Ding, Xibei Jia, Yuting Shen, Zhenzhong Lan, Liwang Zhu, Weiping Liu, Junlin Zhou, Haisheng Liu, Zhong Xin Yu, Pengxin Luo, Donglian Qi, Yunfeng Yan, Junbo Zhao
| Challenge: | Autoregressive (AR) language models dominate modern natural language processing due to strong likelihood-based training objectives and reliable left-to-right decoding. |
| Approach: | They characterize MDLM behavior along two dimensions: parallelism strength and generation order . authors propose a Generate-then-Edit paradigm that mitigates dependency loss . |
| Outcome: | The proposed model improves on tasks that require "backward information" the Generate-then-Edit paradigm improves parallel decoding efficiency while reducing dependency loss. |
Experience is the Teacher: Reusing Atomic Thoughts from LLMs to Improve Medical Dialogue (2026.findings-acl)
Copied to clipboard
Guangya Yu, Hui Luo, Qi Ye, Ruihui Hou, Weiyan Zhang, Mingxi Shang, Xuanwu Li, ChunMing Wang, Tong Ruan
| Challenge: | Recent large reasoning models (LLMs) lack dynamic and diverse thinking capabilities . reusing atomic thoughts provides a practical pathway toward dynamic reasoning . |
| Approach: | They propose a framework that extracts atomic thoughts from teacher models and reuses them to guide reasoning and generate responses. |
| Outcome: | The proposed framework extracts atomic thoughts from teacher models and reuses them to guide reasoning and generate responses. |
SPIDE: Serial and Parallel Intertwined Speculative Decoding (2026.findings-acl)
Copied to clipboard
Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan, Cheng Wang
| Challenge: | Speculative decoding (SD) is a training-free SD framework that orchestrates dynamic alternation combining serial dynamic drafting with parallel draft verification. |
| Approach: | They propose a serial and parallel intertwined speculative DEcoding framework that orchestrates dynamic alternation combining serial dynamic drafting and parallel draft verification. |
| Outcome: | The proposed framework accelerates inference while reducing the LLM usage costs. |
JointLK: Joint Reasoning with Language Models and Knowledge Graphs for Commonsense Question Answering (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing KG-augmented models for commonsense question answering ignore the effectively fusing and reasoning over question context representations and the KG representations. |
| Approach: | They propose a novel model which combines a logical reasoning and a dynamic pruning mechanism to solve these limitations. |
| Outcome: | The proposed model improves existing models and performs interpretable reasoning on the CommonsenseQA and OpenBookQA datasets. |
Improving Question Generation with Multi-level Content Planning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies suggest key phrase selection is essential for question generation, yet it is difficult to connect disjointed phrases into meaningful questions, especially for long context. |
| Approach: | They propose a QG framework that uses multi-level content planning to generate questions from a given context and an answer. |
| Outcome: | The proposed framework outperforms baselines on two popular QG datasets. |
LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to language-based environment manipulation are difficult to generalize across environments. |
| Approach: | They propose a general framework for language-based environment manipulation tasks that can deal with various environments using the same generative language model. |
| Outcome: | The proposed framework achieves new state-of-the-art results on four of the tasks and the execution-guided pre-training strategy brings remarkable improvements on all experimental tasks. |
Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)
Copied to clipboard
Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong
| Challenge: | Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise. |
| Approach: | They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble. |
| Outcome: | The proposed technique improves ensemble performance and robustness against erroneous signals. |
Outcome Accuracy is Not Enough: Aligning the Reasoning Process of Reward Models (2026.acl-long)
Copied to clipboard
Binghai Wang, Yantao Liu, Yuxuan Liu, Tianyi Tang, Shenzhi Wang, Chang Gao, Chujie Zheng, Yichang Zhang, Le Yu, Shixuan Liu, Tao Gui, Qi Zhang, Xuanjing Huang, Bowen Yu, Fei Huang, Junyang Lin
| Challenge: | Recent studies observe a phenomenon where reward models achieve high accuracy on static datasets but fail to generalize effectively during RLHF. |
| Approach: | They propose a method that combines rationale consistency with outcome accuracy to improve performance on RM-Bench and JudgeBench. |
| Outcome: | The proposed method surpasses baselines on RM-Bench and JudgeBench by an average of 5% and improves creative writing tasks by 7%. |
Towards Interpretable Tabular Reasoning: Enhancing LLM Reasoning on Tabular Data with Pre-Constructed Logic Graph (2026.acl-long)
Copied to clipboard
Lirong Gao, Zewei Yu, Zhongrui Yin, Qi Zhang, Yuke Zhu, Bo Zheng, Haobo Wang, Junbo Zhao, Gang Chen, Sheng Guo
| Challenge: | Tabular data is used in fields such as finance and healthcare due to its heterogeneity and complexity. |
| Approach: | They propose a Logic-Graph-Enhanced LLM Reasoning framework that integrates the strengths of tree-based models and LLMs to improve their interpretability. |
| Outcome: | The proposed framework outperforms tree-based models and state-of-the-art LLMs on tabular prediction tasks, achieving superior accuracy and interpretability. |
A Bounding Box is Worth One Token - Interleaving Layout and Text in a Large Language Model for Document Understanding (2025.findings-acl)
Copied to clipboard
Jinghui Lu, Haiyang Yu, Yanjie Wang, Yongjie Ye, Jingqun Tang, Ziwei Yang, Binghong Wu, Qi Liu, Hao Feng, Han Wang, Hao Liu, Can Huang
| Challenge: | Existing methods for integrating spatial layouts with text have limitations . existing methods produce overly long text sequences or lack autoregressive traits of LLMs . |
| Approach: | They introduce Interleaving Layout and Text in a Large Language Model (LayTextLLM) they use OCR-derived text and spatial layouts to integrate with LLMs for document understanding . |
| Outcome: | The proposed model shows an increase in performance in KIE and VQA tasks. |
PLAWBENCH: A Rubric-Based Benchmark for Evaluating LLMs in Real-World Legal Practice (2026.acl-long)
Copied to clipboard
Yuzhen Shi, Huanghai Liu, Yiran HU, Song Gaojie, Xu Xinran, Yubo Ma, Tianyi Tang, Li Zhang, Qingjing Chen, Feng Di, Wenbo Lv, Weiheng Wu, Kexin Yang, Sen Yang, Wei Wang, Rongyao Shi, Qiu Yuanyang, Yuemeng Qi, Zhang Jingwen, Sui Xiaoyu, Yifan Chen, Zhang Yi, An Yang, Bowen Yu, Dayiheng Liu, Junyang Lin, Weixing Shen, Bing Zhao, Charles L. A. Clarke, HU Wei
| Challenge: | Existing benchmarks for large language models (LLMs) are coarse, single-dimensional metrics and do not explicitly assess fine-grained legal reasoning. |
| Approach: | They propose a Practical Law Benchmark to evaluate large language models in real-world legal practice scenarios. |
| Outcome: | The proposed model is based on 850 questions and 13 scenarios with expert-designed evaluation rubrics. |
TestAgent: An Adaptive and Intelligent Expert for Human Assessment (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing adaptive testing methods face several challenges due to mechanized nature of most algorithms and noisy response data. |
| Approach: | They propose to use large language models to enhance adaptive testing through interactive engagement to capture test-takers’ responses and anomalies. |
| Outcome: | The proposed agent achieves more accurate results with 20% fewer questions than state-of-the-art baselines and testers preferred it in speed, smoothness, and other dimensions. |
MARS-Bench: A Multi-turn Athletic Real-world Scenario Benchmark for Dialogue Evaluation (2025.findings-emnlp)
Copied to clipboard
Chenghao Yang, Yinbo Luo, Zhoufutu Wen, Qi Chu, Tao Gong, Longxiang Liu, Kaiyuan Zhang, Jianpeng Jiao, Ge Zhang, Wenhao Huang, Nenghai Yu
| Challenge: | Large Language Models (LLMs) have been widely adopted in real-world dialogue applications, but their robustness is criticized all along. |
| Approach: | They propose to use play-by-play text commentary to build a multi-turn athletic real-world scenario dialogue benchmark to evaluate three critical aspects of multi-turned conversations: ultra multi- turn, interactive multi-twist, and cross-turn tasks. |
| Outcome: | The proposed benchmarks outperform open-source LLMs on three critical aspects of multi-turn conversations: ultra multi-turned, interactive multi- turn, and cross-turn tasks. |
Attribution-Based Analysis and Optimization of Modular Agentic Workflows (2026.findings-acl)
Copied to clipboard
Yingxuan Yang, Bo Huang, Siyuan Qi, Chao Feng, Haoyi Hu, Yuxuan Zhu, Jinbo Hu, Haoran Zhao, Ziyi He, Xiao Liu, ZongYu Wang, Muning Wen, Lin Qiu, Xuezhi Cao, Xunliang Cai, Yong Yu, Weinan Zhang
| Challenge: | Large Language Models (LLMs) have driven the rise of agentic workflows . yet, how can we attribute performance gains to individual upgrades and their interactions? |
| Approach: | They propose a game-theoretic framework that models component upgrades as players and evaluates component coalitions to compute Shapley values. |
| Outcome: | The proposed framework provides interaction-aware attribution and recommendation for model allocation under a fixed workflow structure. |
On Large Language Models’ Hallucination with Regard to Known Facts (2024.naacl-long)
Copied to clipboard
Che Jiang, Biqing Qi, Xiangyu Hong, Dayuan Fu, Yang Cheng, Fandong Meng, Mo Yu, Bowen Zhou, Jie Zhou
| Challenge: | Large language models are successful in answering factoid questions but are also prone to hallucination. |
| Approach: | They propose self-reporting to the model when faced with such limitations. |
| Outcome: | The proposed classifier can detect hallucinations with an 88% success rate and can be used to answer factoid questions with correct answer knowledge. |
RESIN-11: Schema-guided Event Prediction for 11 Newsworthy Scenarios (2022.naacl-demo)
Copied to clipboard
Xinya Du, Zixuan Zhang, Sha Li, Pengfei Yu, Hongwei Wang, Tuan Lai, Xudong Lin, Ziqi Wang, Iris Liu, Ben Zhou, Haoyang Wen, Manling Li, Darryl Hannan, Jie Lei, Hyounghun Kim, Rotem Dror, Haoyu Wang, Michael Regan, Qi Zeng, Qing Lyu, Charles Yu, Carl Edwards, Xiaomeng Jin, Yizhu Jiao, Ghazaleh Kazeminejad, Zhenhailong Wang, Chris Callison-Burch, Mohit Bansal, Carl Vondrick, Jiawei Han, Dan Roth, Shih-Fu Chang, Martha Palmer, Heng Ji
| Challenge: | Existing methods for event prediction are incomplete and noisy. |
| Approach: | They propose to use news-related event schemas to extract newsworthy events . they build a demo website and include a video demonstrating the framework . |
| Outcome: | The proposed framework can be applied to a wide variety of newsworthy scenarios. |
RePair: Automated Program Repair with Process-based Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Commercial-scale language models (LMs) have taken APR to unprecedented levels, but they are limited by parameters and humans interact with them through explicit prompts. |
| Approach: | They propose a method that utilizes process supervision to improve program repair by allowing users to input feedback from compilers and test cases. |
| Outcome: | The proposed method outperforms large outcome-based generation methods and is inspired by strategies used in programming competitions. |
Semi-Supervised Dialogue Policy Learning via Stochastic Reward Estimation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for dialogue policy optimization do not provide sufficient supervision signals at the end of dialogues. |
| Approach: | They propose to learn from state-action pairs of an optimal policy to provide turn-by-turn rewards. |
| Outcome: | The proposed approach outperforms competitive policy learning baselines on a benchmark multi-domain dataset. |
CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculation (2025.findings-acl)
Copied to clipboard
Guangya Yu, Yanhao Li, Zongying Jiang, Yuxiong Jin, Li Dai, Yupian Lin, Ruihui Hou, Weiyan Zhang, Yongqi Fan, Qi Ye, Jingping Liu, Tong Ruan
| Challenge: | Medical quality control indicators are essential to assess the qualifications of healthcare institutions for medical services. |
| Approach: | They propose a Chinese electronic medical records-based dataset for MQCIC and propose CF-IR method that disentangles clinical fact verification and inferential rule reasoning actions. |
| Outcome: | The proposed method outperforms Chain-of-Thought methods on 20 representative LLMs, covering general and medical models. |
HIPO: A Hierarchical Prompt Optimization Framework with Task Awareness and Fine-Grained Debugging (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for prompt optimization apply the same prompt across all samples . existing methods ignore variation in sample difficulty . |
| Approach: | They propose a framework that shifts the paradigm from dataset-level to sample-level optimization. |
| Outcome: | The proposed framework outperforms baselines on 27 tasks and reduces API calls, token consumption and overall cost by 1.2 to 80. |
Latent Reasoning for Low-Resource Question Generation (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to multihop question generation require extensive data collection and decomposition. |
| Approach: | They propose a generative approach that optimizes the two-phase model without question decomposition data. |
| Outcome: | The proposed approach outperforms baselines on HOTPOTQA, a benchmark multi-hop question answering dataset. |
CA-GAR: Context-Aware Alignment of LLM Generation for Document Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent techniques such as Generation-Augmented Retrieval (GAR) and Generative Document Retrieleval (GDR) leverage LLMs to enhance retrieval performance but face key challenges: GAR’s generated content may not always align with the target document corpus, while GDR limits the generative capacity of LLM. |
| Approach: | They propose a Context-Aware Generation-Augmented Retrieval approach which integrates corpus information into their generation process. |
| Outcome: | Experimental results show that CA-GAR outperforms existing methods on seven tasks and four non-English languages. |
HieRec: Hierarchical User Interest Modeling for Personalized News Recommendation (2021.acl-long)
Copied to clipboard
| Challenge: | Existing news recommendation methods learn a single user embedding for each user from their previous behaviors to represent their overall interest. Existing methods only learn 'one' embeddable representation vectors to model user interest. |
| Approach: | They propose a news recommendation method with hierarchical user interest modeling that captures user interest in news rather than a single user embedding. |
| Outcome: | The proposed method can better capture multi-grained user interest in news. |
PKAD: Pretrained Knowledge is All You Need to Detect and Mitigate Textual Backdoor Attacks (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current defense methods can be classified into inference-time and training-time ones based on their execution phase. |
| Approach: | They propose a two-stage poison detection strategy using pre-trained language models to detect poisoned samples before model training. |
| Outcome: | The proposed method achieves better performance than current methods more quickly and with fewer training costs. |
All Information is Valuable: Question Matching over Full Information Transmission Network (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods for question matching only transmit one kind of information while failing to utilize both kinds of information simultaneously. |
| Approach: | They propose a question matching network that can transmit both representation and interactive information together in a simultaneous fashion. |
| Outcome: | The proposed approach outperforms strong baseline models on two standard benchmarks. |
Eval-RAR: Evaluation-Driven Retrieval-Augmented Reasoning via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for retrieval-augmented generation fail to provide explicit supervision for internal reasoning process. |
| Approach: | They propose an Evaluation-driven Retrieval-Augmented Reasoning framework that uses reinforcement learning and a fine-grained evaluation reward to optimize the process. |
| Outcome: | Eval-RAR outperforms existing methods on QA benchmarks on seven single-hop and multi-hop tasks. |
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation (2021.acl-demo)
Copied to clipboard
Weizhen Qi, Yeyun Gong, Yu Yan, Can Xu, Bolun Yao, Bartuer Zhou, Biao Cheng, Daxin Jiang, Jiusheng Chen, Ruofei Zhang, Houqiang Li, Nan Duan
| Challenge: | Existing models for pre-training are not convenient for users to find and set them up. |
| Approach: | They propose to extend ProphetNet into other domains and languages by pre-training models . they pre-train a cross-lingual generation model ProphetNet-Multi and a Chinese generation model . |
| Outcome: | The proposed models achieve new state-of-the-art on 10 benchmarks. |
Controllable Contamination Detection for Reliable LLM Evaluation with Statistical Guarantees (2026.acl-long)
Copied to clipboard
Zheng Zhang, Qi Liu, Siyuan Liang, Ning Li, Zirui Hu, Weibo Gao, Rui Li, Zhenya Huang, Leszek Rutkowski, Baosheng Yu, Dacheng Tao
| Challenge: | Existing training data detectors fail to detect clean samples from contaminated test sets . existing methods fail to identify clean samples due to black-box nature of LLMs . |
| Approach: | They propose a framework that detects and filters contaminated evaluation data . they propose 'failure detection' to reduce the proportion of contaminated samples mistakenly retained . |
| Outcome: | The proposed framework reduces false discovery rate (FDR) under valid FDR control while maintaining evaluation consistency. |
NL2Lean: Translating Natural Language into Lean 4 through Multi-Aspect Reinforcement Learning (2025.emnlp-main)
Copied to clipboard
Yue Fang, Shaohan Huang, Xin Yu, Haizhen Huang, Zihan Zhang, Weiwei Deng, Furu Wei, Feng Sun, Qi Zhang, Zhi Jin
| Challenge: | Existing formal proof assistants rely on instruction tuning and lack fine-grained structural and semantic alignment. |
| Approach: | They propose a reinforcement learning framework that enables LLMs to translate natural language into formal language such as Lean 4 . they use a model with basic translation ability to refine the model's reinforcement learning . |
| Outcome: | The proposed method outperforms baseline models on NL-to-Lean 4 tasks. |
SciCustom: A Framework for Custom Evaluation of Scientific Capabilities in Large Language Models (2026.acl-long)
Copied to clipboard
Yiyang Gu, Junwei Yang, Junyu Luo, Ye Yuan, Bin Feng, Yingce Xia, Shufang Xie, Kaili Liu, Bohan Wu, Qi Shi, Haoran Li, Beier Xiao, Zhiping Xiao, Xiao Luo, Weizhi Zhang, Philip S. Yu, Zequn Liu, Ming Zhang
| Challenge: | Existing evaluations of large language models fail to reflect fine-grained capabilities . existing benchmarks are manually curated or domain-generic, limiting scalability and alignment with real use cases. |
| Approach: | They propose a framework that allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific scientific capabilities in LLMs. |
| Outcome: | The proposed framework reveals fine-grained differences in scientific capabilities that standard benchmarks overlook . it allows custom construction of benchmarks from large-scale scientific data to evaluate application-specific capabilities in LLMs. |
Generalizable and Explainable Dialogue Generation via Explicit Action Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Conditioned response generation for task-oriented dialogues implicitly optimizes task completion and language quality. |
| Approach: | They propose to learn natural language actions that represent utterances as a span of words. |
| Outcome: | The proposed approach outperforms latent action baselines on a multi-domain dataset. |
MMoE: Enhancing Multimodal Models with Mixtures of Multimodal Interaction Experts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal models focus on the correspondence between images and text, but this only covers a subset of real-world interactions. |
| Approach: | They propose an approach to enhance multimodal models by training separate expert models for each type of interaction, such as redundancy present in both modalities, uniqueness in one modality, or synergy that emerges when both . modality is used to capture overlaps in semantic content between images and text, making a strong multi-view redundancies assumption. |
| Outcome: | The proposed approach improves on a sarcasm detection and humor detection task. |
Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory (2026.acl-long)
Copied to clipboard
Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li, Jiaxi Zhou, Hualei Wang, Haohua Wang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang
| Challenge: | Existing methods for retrieving historical messages are based on similarity-based mechanisms. |
| Approach: | They propose a system that integrates System-1 similarity search with a complementary System-2 mechanism, termed Global Selection. |
| Outcome: | The proposed framework achieves state-of-the-art on long-term memory benchmarks and 93.9 on LoCoMo and 91.6 on LongMemEval-S. |
TSGP: Two-Stage Generative Prompting for Unsupervised Commonsense Question Answering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on acquiring relevant knowledge by retrieving external knowledge bases and fine-tuning pre-trained models. |
| Approach: | They propose a two-stage prompt-based unsupervised commonsense question answering framework that leverages implicit knowledge stored in PrLMs to generate knowledge for questions with unlimited types and possible candidate answers independent of specified choices. |
| Outcome: | The proposed framework significantly improves the reasoning ability of language models in unsupervised settings. |
GLGE: A New General Language Generation Evaluation Benchmark (2021.findings-acl)
Copied to clipboard
Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan
| Challenge: | Multi-task benchmarks focus on a range of Natural Language Understanding (NLU) tasks without considering the Natural Language Generation (NLG) models. |
| Approach: | They propose a multi-task benchmark for evaluating the generalization capabilities of NLG models across eight language generation tasks. |
| Outcome: | The proposed benchmarks are based on GLUE and Su-perGLUE for English and several other languages. |
AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation (2026.findings-acl)
Copied to clipboard
Wentao Shi, Yu Wang, Yuyang Zhao, Yuxin Chen, Fuli Feng, Xueyuan Hao, Xi Su, Qi GU, Hui Su, Xunliang Cai, Xiangnan He
| Challenge: | Existing approaches to verify agent behaviors in complex environments rely on rule-based verifiers or LLM-as-a-Judge models. |
| Approach: | They propose a benchmark to evaluate Agent-as-a-Judge across three domains . the benchmark covers search, data systems, and graphical user interfaces - with 155 tasks and 516 trajectories . |
| Outcome: | The proposed benchmark outperforms existing benchmarks in search, data systems, and GUI domains while revealing open challenges in agent-based verification. |
GRIT: A Dataset of Group Reference Recognition in Italian (2024.lrec-main)
Copied to clipboard
| Challenge: | a task of automatically recognizing group references has not yet gained much attention within NLP. |
| Approach: | They propose a large-scale dataset for automatic group reference recognition in italian . they verify the validity of the task using a fine-tuned BERT model . |
| Outcome: | The proposed dataset proves that it can be applied to political text analysis and social media analysis. |
On the token distance modeling ability of higher RoPE attention dimension (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on extending the context length of language models based on Rotary position embedding (RoPE) has shown promising results in capturing longer-range contextual information. |
| Approach: | They propose to use a hidden dimension of an attention head to investigate its contribution to capturing long-distance dependencies. |
| Outcome: | The proposed model can capture long-distance dependencies by extending the attention of a particular dimension of an attention head. |
GEM: Gestalt Enhanced Markup Language Model for Web Understanding via Render Tree (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing models that use plain HTMLs do not include crucial visual information in the rendered web. |
| Approach: | They propose a Gestalt Enhanced Markup Language Model for hosting visual information without visual input. |
| Outcome: | The proposed model can handle multiple downstream tasks without visual input. |
Enhancing Large Vision-Language Models with Ultra-Detailed Image Caption Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing pipelines for generating high-quality, ultra-detailed image captions are limited by the scarcity of image caption data. |
| Approach: | They propose a pipeline for generating high-quality, ultra-detailed image captions that integrates both pre-processing and post-processor stages. |
| Outcome: | The proposed pipeline improves LVLMs' perception and cognitive abilities across multiple vision-language benchmarks. |
ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization (2025.findings-emnlp)
Copied to clipboard
Zhensheng Jin, Xinze Li, Yifan Ji, Chunyi Peng, Zhenghao Liu, Qi Shi, Yukun Yan, Shuo Wang, Furong Peng, Ge Yu
| Challenge: | Existing methods to train LLMs suffer from overthinking, leading to lengthy reasoning traces . Existing approaches to train large language models suffer from this problem . |
| Approach: | They propose a method to combine multiple reasoning chains for training LLMs . they use stepwise exploration and long-short switched sampling to evaluate reasoning paths . |
| Outcome: | The proposed method reduces reasoning lengths by approximately 30-50% . it also maintains or improves reasoning accuracy compared to baselines . |
Divide and Conquer: Legal Concept-guided Criminal Court View Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for creating rationales for criminal cases do not pay enough attention to the important legal concepts. |
| Approach: | They propose a legal concept-guided court view generation framework that generates rationales based on predicted legal concepts . they first divide the court view into sub-views, then employ a solver and verifier to generate and select rationale. |
| Outcome: | The proposed model generates coherent and coherent court views on a real-world criminal case dataset. |
EgoMemory: Memory-Augmented Personalized Retrieval for Long-Context Egocentric Video (2026.findings-acl)
Copied to clipboard
Yuanmin Tang, Jue Zhang, Xiaoting Qin, Jing Yu, Meikang Qiu, Gaopeng Gou, Gang Xiong, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang, Qi Wu
| Challenge: | Existing egocentric video datasets do not support the personalization and long-context reasoning required for episodic memory retrieval. |
| Approach: | They propose a benchmark framework that uses MLLMs and reflective Chain-of-Thought to ground user queries in personal memory explicitly. |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks on three benchmarks . it can be used to generate detailed target video descriptions in long-context contexts based on user-specific object annotations enriched with user-specified object annotation data . |
Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable capabilities in various tasks, but may rely on dataset biases as shortcuts for prediction. |
| Approach: | They propose to use a test suite to evaluate the impact of shortcuts on LLMs' performance. |
| Outcome: | The proposed test suite incorporates six shortcut types, five evaluation metrics, and four prompting strategies. |
Long-context Language Models Fail in Basic Retrieval Tasks Without Sufficient Reasoning Steps (2025.findings-emnlp)
Copied to clipboard
| Challenge: | despite their extensive context window, long-context language models fail in some basic cases . a recent study shows that long-cot methods are not necessary for long-constituency tasks . |
| Approach: | a new study evaluates long-context language models with a large context window . the authors propose a method that can be well addressed with arbitrary reasoning steps . |
| Outcome: | The proposed methods are well addressed with a sufficient number of reasoning steps, guided by specific CoT prompts. |
Pretraining the Noisy Channel Model for Task-Oriented Dialogue (2021.tacl-1)
Copied to clipboard
| Challenge: | Current research on task-oriented dialogue models suffers from the explaining-away effect, manifested in models that favor short and generic responses. |
| Approach: | They propose to factorize the dialogue task into two models, the distribution of the context given the response, and the prior for the response itself, using Bayes' theorem. |
| Outcome: | The proposed model mitigates the explaining-away effect and allows the principled incorporation of large pretrained models for the response prior. |
Diversify Question Generation with Retrieval-Augmented Style Transfer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing question generation systems focus on the internal knowledge within the textual passage or the semantic word space for diverse content planning. Existing solutions focus on relying on the knowledge of the text and the semantic words, but have not considered the potential of external knowledge for expression diversity. |
| Approach: | They propose a framework for Retrieval-Augmented Style Transfer that utilizes the style of diverse templates for question generation. |
| Outcome: | The proposed framework outperforms baselines on diversity while being comparable in terms of consistency scores. |
MCP-Guard: A Multi-Stage Defense-in-Depth Framework for Securing Model Context Protocol in Agentic AI (2026.findings-acl)
Copied to clipboard
Wenpeng Xing, Zhonghao Qi, Yupeng Qin, Yilin Li, Caini Chang, Jiahui Yu, Changting Lin, Zhenzhen Xie, Meng Han
| Challenge: | Large Language Models (LLMs) are vulnerable to jailbreak, authors say . authors propose a robust, layered defense architecture designed for LLM–tool interactions . |
| Approach: | They propose a robust, layered defense architecture designed for LLM–tool interactions . they propose XCP-Guard, which employs a three-stage detection pipeline . |
| Outcome: | The proposed model achieves 96.01% accuracy in identifying adversarial prompts . the model is based on a three-stage detection pipeline that balances efficiency with accuracy . |
MMLU-ProX: A Multilingual Benchmark for Advanced Large Language Model Evaluation (2025.emnlp-main)
Copied to clipboard
Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, Felix Juefei-Xu, Foutse Khomh, Osamu Yoshie, Qingyu Chen, Douglas Teodoro, Nan Liu, Randy Goebel, Lei Ma, Edison Marrese-Taylor, Shijian Lu, Yusuke Iwasawa, Yutaka Matsuo, Irene Li
| Challenge: | Existing large language model evaluation benchmarks focus on English, while current multilingual tasks lack parallel questions that specifically assess cross-lingual reasoning abilities. |
| Approach: | They propose a comprehensive benchmark covering 29 languages, built on an English benchmark. |
| Outcome: | The MMLU-ProX is a comprehensive benchmark covering 29 languages, built on an English benchmark. |
TRAC: Teacher-Guided Token Reward with Adaptive Calibration for Robust Policy Optimization (2026.acl-long)
Copied to clipboard
| Challenge: | Current reward models for reinforcement learning (RL) rely on outcome rewards that propagate a single scalar value across all tokens based on final correctness. |
| Approach: | They propose a framework that derives dense token-level supervision from LLMs . they use a multi-granularity calibration mechanism to modulate teacher influence . |
| Outcome: | The proposed framework evaluates teacher reliability across problem-level expertise, trajectory-level discrimination, and token-level confidence. |
Fin-STAR: Structure-as-Semantics to Resolve Implicitness in Financial Retrieval (2026.findings-acl)
Copied to clipboard
Yu Zou, Yan Chen, Lida He, Qi Zhou, Xiaorui Zhou, Aixi Zhong, Yi Wang, Wei Li, Qingyu Wang, Jiatao Li, Wei Gong, Jialei Zeng, Jingmei Zhao, Ke Jiang, Qing Li
| Challenge: | Existing Retrieval-Augmented Generation systems treat structure as a physical navigational skeleton rather than intrinsic semantic knowledge. |
| Approach: | They propose a framework that redefining hierarchy as intrinsic semantics and uses snippets to enrich hierarchical lineage. |
| Outcome: | The proposed framework outperforms state-of-the-art hierarchical and graph-based benchmarks on FinTierQA Gold. |
CENTAUR: Bridging the Impossible Trinity of Privacy, Efficiency, and Performance in Privacy-Preserving Transformer Inference (2025.acl-long)
Copied to clipboard
Jinglong Luo, Guanzhong Chen, Yehong Zhang, Shiyu Liu, Hui Wang, Yue Yu, Xun Zhou, Yuan Qi, Zenglin Xu
| Challenge: | Existing privacy-preserving Transformer Inference frameworks suffer from high computational overhead and performance losses. |
| Approach: | They propose a framework that integrates random permutations and SMPC to address the "impossible trinity" CENTAUR resists diverse data reconstruction attacks and boosts inference speed by 5.030.4 times . |
| Outcome: | CENTAUR achieves an unprecedented balance between privacy, efficiency, and performance. |
EcomScriptBench: A Multi-task Benchmark for E-commerce Script Planning via Step-wise Intention-Driven Product Association (2025.acl-long)
Copied to clipboard
Weiqi Wang, Limeng Cui, Xin Liu, Sreyashi Nag, Wenju Xu, Chen Luo, Sheikh Muhammad Sarwar, Yang Li, Hansu Gu, Hui Liu, Changlong Yu, Jiaxin Bai, Yifan Gao, Haiyang Zhang, Qi He, Shuiwang Ji, Yangqiu Song
| Challenge: | Goal-oriented script planning is used by humans to plan for typical activities . however, this capability remains underexplored due to several challenges . |
| Approach: | They propose a framework that enables product-enriched scripts by associating products with each step based on the semantic similarity between the actions and their purchase intentions. |
| Outcome: | The proposed framework can generate product-enriched scripts from 2.4 million scripts . human annotations are conducted to provide gold labels for a sampled subset . |
RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translation (2025.findings-emnlp)
Copied to clipboard
Tianjiao Li, Mengran Yu, Chenyu Shi, Yanjun Zhao, Xiaojing Liu, Qi Zhang, Xuanjing Huang, Qiang Zhang, Jiayin Wang
| Challenge: | Using reinforcement learning from human feedback, large language models perform poorly when applied to colloquial subtitle translation tasks. |
| Approach: | They propose an adversarial training framework that iteratively updates the offline reward model and the online LLM to improve training outcomes. |
| Outcome: | The proposed training framework significantly improves upon translation baselines. |
UniRAG: Unified Query Understanding Method for Retrieval Augmented Generation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing query augmentation methods face knowledge update lag and hallucinations in large language models (LLMs) Existing methods face two key challenges: (1) separation of query augmented and encoding tasks, which hinders information sharing and introduces cumulative errors; (2) difficulty of selecting optimal augmentation strategy for different scenarios. |
| Approach: | They propose a unified framework for query understanding in RAG that integrates internal and external knowledge to enhance query augmentation and encoding tasks. |
| Outcome: | The proposed framework outperforms traditional query augmentation methods in five knowledge-intensive benchmark tasks in both closed and open domain question answering. |
EconProver: Towards More Economical Test-Time Scaling for Automated Theorem Proving (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have recently advanced the field of Automated Theorem Proving (ATP) Existing cost analyses regulate only the number of sampling passes, ignoring the substantial disparities in sampling costs. |
| Approach: | They propose to integrate two complementary methods into a unified EconRL pipeline to increase pass rates under constrained sampling passes. |
| Outcome: | The proposed method reduces token usage and sample passes while maintaining the original performance. |
Explanation Graph Generation via Generative Pre-training over Synthetic Graphs (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for explanation graph generation are limited due to the large number of datasets available. |
| Approach: | They propose a text-to-graph generative task to pre-train a model to bridge the text-graph gap. |
| Outcome: | The proposed framework surpasses all baseline systems with remarkable margins on ExplaGraphs and CommonsenseQA. |
ESF: Efficient Sensitive Fingerprinting for Black-Box Tamper Detection of Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly utilized in diverse applications, including code generation, legal document analysis, medical diagnosis, and decision-making. |
| Approach: | They propose a fingerprinting method tailored for black-box tamper detection of large language models. |
| Outcome: | The proposed method detects tampering with a 99.2% detection rate using 5 fingerprint samples across state-of-the-art LLMs. |
Logic-level Evidence Retrieval and Graph-based Verification Network for Table-based Fact Verification (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods leverage programs that contain rich logical information to enhance the verification process. |
| Approach: | They propose a table-based fact verification task as an evidence retrieval framework . they retrieve logic-level program-like evidence from the given table and a statement as supplementary evidence for the table . |
| Outcome: | The proposed method is able to retrieve logic-level program-like evidence from a table and a statement as supplementary evidence for the table. |
ChartMind: A Comprehensive Benchmark for Complex Real-world Multimodal Chart Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Chart question answering (CQA) is a multimodal task for evaluating the reasoning capabilities of vision-language models. |
| Approach: | They propose a chart question answering benchmark that incorporates multilingual contexts and supports open-domain textual outputs. |
| Outcome: | The proposed framework outperforms the previous three common CQA paradigms: instruction-following, OCR-enhanced, and chain-of-thought. |
Measuring Human Contribution in AI-Assisted Content Generation (2026.acl-long)
Copied to clipboard
Yueqi Xie, Tao Qi, Jingwei Yi, Xiyuan Yang, Ryan Whalen, Junming Huang, Qian Ding, Yu Xie, Xing Xie, Fangzhao Wu
| Challenge: | generative AI has created a new way to generate content with humans . varying degrees of human contribution in content generation poses significant challenges for the delineation of originality . |
| Approach: | They propose a framework to measure human contribution in AI-assisted content generation by calculating mutual information between human input and AI-aided output relative to self-information of AI-assist output. |
| Outcome: | The proposed measure discriminates between varying degrees of human contribution across multiple creative domains and is validated in real-world applications. |
Is Cognition Consistent with Perception? Assessing and Mitigating Multimodal Knowledge Conflicts in Document Understanding (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) have shown impressive capabilities in document understanding due to different types of annotation noise in training. |
| Approach: | They propose a method to reduce C&P knowledge conflicts across all tested MLLMs . they propose to use annotation noise to train models to understand document content . |
| Outcome: | The proposed method reduces C&P knowledge conflicts across all tested MLLMs and enhances their performance in both cognitive and perceptual tasks. |
Answering Cross-Dimensional Geometric Visual Questions by Multi-constraint Spatial Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for solving complex visual questions are limited in their ability to represent in a cross-dimensional space. |
| Approach: | They propose a method that can answer complex visual questions using cross-dimensional reasoning. |
| Outcome: | The proposed method can answer complex visual questions in 2D to 3D space with great application value. |
from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies have exposed the risk of Large Language Models (LLMs) generating harmful content by jailbreak attacks. |
| Approach: | They propose a framework that exploits AdVersArial meTAphoR to induce LLMs to calibrate harmful metaphors for jailbreaking. |
| Outcome: | The proposed framework can successfully jailbreak Large Language Models (LLMs) by leveraging the AdVersArial meTAphoR (AVATAR) framework achieves state-of-the-art attack success rate across multiple advanced LLMs. |
DocHieNet: A Large and Diverse Dataset for Document Hierarchy Parsing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for document hierarchy parsing are limited due to the small scale and inconsistency of datasets. |
| Approach: | They propose a document hierarchy parsing dataset to compensate for the data scarcity problem and propose 'dHP' framework to grasp fine-grained text content and coarse-grounded pattern at layout element level. |
| Outcome: | The proposed framework grasps both fine-grained text content and coarse-grounded pattern at layout element level, enhancing the capacity of pre-trained text-layout models in handling multi-page and multi-level challenges. |
High-Quality Medical Dialogue Synthesis for Improving EMR Generation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for generating EMRs from doctor-patient dialogues produce rigid and repetitive dialogues. |
| Approach: | They propose a framework that integrates Intent Graph Planning, Dual-Agent Simulation and Rule-Reward Quality Control to generate realistic doctor-patient dialogues. |
| Outcome: | The proposed framework significantly enhances realism, diversity and downstream EMR quality, reducing physician editing efforts. |
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) often struggle to accurately express factual knowledge, especially in cases where the knowledge boundaries are ambiguous. |
| Approach: | They propose a framework that leverages Uncertainty estimations to represent knowledge boundaries and incorporates these representations into prompts for LLMs to Align with factual knowledge. |
| Outcome: | The proposed framework significantly improves the LLMs’ capacities to confidently answer known questions and refuse unknown questions on both in-domain and out-of-domain tasks. |
RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL (2022.emnlp-main)
Copied to clipboard
Jiexing Qi, Jingyao Tang, Ziwei He, Xiangpeng Wan, Yu Cheng, Chenghu Zhou, Xinbing Wang, Quanshi Zhang, Zhouhan Lin
| Challenge: | Experimental results show RASAT can leverage a variety of relational structures while inheriting the pretrained parameters from the T5 model. |
| Approach: | They propose a Transformer seq2seq architecture augmented with relation-aware self-attention that leverages relational structures while inheriting pretrained parameters from the T5 model. |
| Outcome: | The proposed model can leverage relational structures while inheriting pretrained parameters from the T5 model effectively. |
Nirvana: A Specialized Generalist Model With Task-Aware Memory Mechanism (2026.acl-long)
Copied to clipboard
Yuhua Jiang, Shuang Cheng, Yihao Liu, Ermo Hua, Che Jiang, Weigao Sun, Yu Cheng, Feifei Gao, Biqing Qi, Bowen Zhou
| Challenge: | Existing models lack task-guided specialized memory mechanisms . specialized generalist models excel at general language tasks but struggle in specialized domains. |
| Approach: | They propose a specialized generalist model with specialized memory and updater that can optimize for specialized domains. |
| Outcome: | The proposed model matches or surpasses baselines on general benchmarks and achieves lowest perplexity across specialized domains. |
Few-shot Intent Classification and Slot Filling with Retrieved Examples (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for few-shot learning are based on labeled examples, but they are non-trivial . few-sshot learning is challenging due to the imbalance in the amount of data between the source and target domains. |
| Approach: | They propose retrieval-based methods for intent classification and slot filling tasks . they use a batch-softmax objective to learn similar contextualized representations for spans . |
| Outcome: | The proposed method outperforms previous systems on the CLINC and SNIPS benchmarks. |
AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following (2026.acl-long)
Copied to clipboard
Yun He, Wenzhe Li, Hejia Zhang, Songlin Li, Karishma Mandyam, Sopan Khosla, Yuanhao Xiong, Nanshu Wang, Xiaoliang Peng, Beibin Li, Shengjie Bi, Shishir G Patil, Qi Qi, Shengyu Feng, Julian Katz-Samuels, Richard Yuanzhe Pang, Sujan Kumar Gonugondla, Hunter Lang, Yue Yu, Yundi Qian, Maryam Fazel-Zarandi, Licheng Yu, Amine Benhalloum, Hany Hassan Awadalla, Manaal Faruqui
| Challenge: | Recent advances in large language models (LLMs) have shown impressive performance on a range of tasks, yet advanced instruction following (IF) remains a significant challenge. |
| Approach: | They propose a benchmark that features over 1,600 prompts and expert-curated rubrics that assess LLMs’ ability to follow complex, multi-turn, and system-level instructions. |
| Outcome: | The proposed framework improves instruction-following abilities of large language models, achieving a 6.7% gain on AdvancedIF and strong results on public benchmarks. |
LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)
Copied to clipboard
Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun
| Challenge: | Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size. |
| Approach: | They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding. |
| Outcome: | The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models. |
QA-Driven Zero-shot Slot Filling with Weak Supervision Pretraining (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods to predict slots and their values do not encode enough semantic information, limiting the models’ zero-shot capability. |
| Approach: | They propose a QA-driven slot filling model which extracts slot-filler spans from utterances with a span-based QA model. |
| Outcome: | The proposed model outperforms baselines by over 5% on the SNIPS benchmark. |
When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviors (2026.acl-long)
Copied to clipboard
| Challenge: | Existing metrics fail to distinguish mandatory behaviors required for task success from non-mandatory patterns that reflect a model’s autonomous preferences. |
| Approach: | They propose to use response pattern similarity and action graph similarity to isolate non-mandatory behaviors from mandatory behaviors. |
| Outcome: | Evaluating 18 models from 8 providers on -Bench and 2-Bench against Claude Sonnet 4.5, the authors find that within-family model pairs score 5.9 pp higher in response pattern similarity and action graph similarity . |
Learn to Combine Linguistic and Symbolic Information for Table-based Fact Verification (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification lack attention to combine linguistic and symbolic information. |
| Approach: | They propose a graph-based reasoning approach that learns to combine linguistic and symbolic information effectively. |
| Outcome: | The proposed method can combine linguistic and symbolic information effectively. |
SafeScientist: Enhancing AI Scientist Safety for Risk-Aware Scientific Discovery (2025.emnlp-main)
Copied to clipboard
Kunlun Zhu, Jiaxun Zhang, Ziheng Qi, Nuoxing Shang, Zijia Liu, Peixuan Han, Yue Su, Haofei Yu, Jiaxuan You
| Challenge: | Recent advances in large language model (LLM) agents have significantly accelerated scientific discovery automation, yet raised critical ethical and safety concerns. |
| Approach: | They propose a framework to enhance safety and ethical responsibility in AI-driven scientific exploration. |
| Outcome: | The proposed framework significantly improves safety performance by 35% compared to traditional frameworks. |
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models are pre-trained and distilled on general corpus like Wikipedia, which has gaps with the news domain and may be suboptimal for news intelligence. |
| Approach: | They propose a method to distill existing language models on Wikipedia to enable efficient news intelligence. |
| Outcome: | The proposed model can be used to build and test a news intelligence application on Wikipedia and Wikipedia. |