Papers with verification
Entity-aware Cross-lingual Claim Detection for Automated Fact-checking (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing work on verifiable claims detection is focused on monolingual solutions . identifying and validating claims related to global concerns requires a fact-checking pipeline capable of processing claims written in multiple languages. |
| Approach: | They propose an entity-aware cross-lingual claim detection model that generalizes well to handle multilingual claims. |
| Outcome: | The proposed model shows consistent performance gains across 27 languages and robust knowledge transfer between languages seen and unseen during training. |
Claim Verification in the Age of Large Language Models: A Survey (2026.acl-srw)
Copied to clipboard
| Challenge: | Recent election cycles have seen a large number of false information spread across social media and news platforms. |
| Approach: | They propose a framework for automated claim verification using Large Language Models and Retrieval Augmented Generation. |
| Outcome: | The proposed frameworks are based on large-scale models and new methods such as Retrieval Augmented Generation (RAG). |
Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing (2025.acl-industry)
Copied to clipboard
| Challenge: | a new language model that supports 512K context lengths addresses practical limitations in long-context training . a competitive 35% score on 512k-token BABILong tasks without RAG or task-specific tuning is achieved . |
| Approach: | They present a language model that supports 512K-token context length . they evaluated its long-context learning performance on three benchmarks . |
| Outcome: | The model outperforms open-source models on three long-context benchmarks . it achieves a competitive 35% score on 512K-token BABILong tasks without RAG or fine-tuning . |
OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure (2025.tacl-1)
Copied to clipboard
| Challenge: | Autoregressive language models generate one token in one step, limiting inference efficiency . Existing methods do not adapt to different situations to maximize acceptance length . speculative decoding has shown great potential for lossless acceleration . |
| Approach: | They propose an algorithm to construct adaptive and scalable draft trees for autoregressive language models. |
| Outcome: | Experimental results show that OPT-Tree outperforms existing draft trees and achieves speed-up ratio of up to 3.2 compared with autoregressive decoding. |
ParsFEVER: a Dataset for Farsi Fact Extraction and Verification (2021.starsem-1)
Copied to clipboard
Majid Zarharan, Mahsa Ghaderan, Amin Pourdabiri, Zahra Sayedi, Behrouz Minaei-Bidgoli, Sauleh Eetemadi, Mohammad Taher Pilehvar
| Challenge: | Existing methods for fact-checking and verification require large amounts of annotated data, but this is limited to low-resource languages. |
| Approach: | They present a first publicly available Farsi dataset for fact extraction and verification . they use the construction procedure of the standard English dataset for the task . |
| Outcome: | The proposed dataset improves on the standard English dataset and is available on github. |
A Browser-based Open Source Assistant for Multimodal Content Verification (2026.eacl-demo)
Copied to clipboard
Rosanna Milner, Michael Foster, Twin Karmakharm, Olesya Razuvayevskaya, Valentin Porcellini, Denis Teyssou, Ian Roberts, Kalina Bontcheva
| Challenge: | Disinformation and advanced generative AI content pose a significant challenge for journalists and fact-checkers who must rapidly verify digital media. |
| Approach: | They propose to integrate a browser-based tool that automatically extracts content from a suite of backend NLP classifiers and presents actionable credibility signals and AI-generation likelihood in an easy-to-digest format. |
| Outcome: | The Verification Assistant is a browser-based tool that extracts content and routes it to a suite of backend NLP classifiers, presenting actionable credibility signals, AI-generation likelihood, and other verification advice in an easy-to-digest format. |
Toward Reliable Clinical Coding with Language Models: Verification and Lightweight Adaptation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing methods for clinical code verification fail to account for hierarchical misalignments . standardized coding systems such as ICD-10-CM1 ensure consistency across medical records. |
| Approach: | They propose to use prompt engineering and small-scale fine-tuning to improve accuracy without the computational overhead of search-based methods. |
| Outcome: | The proposed task is a standalone task and a pipeline component to address hierarchical near-miss errors without the computational overhead of search-based methods. |
Table-based Fact Verification with Self-adaptive Mixture of Experts (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing research focuses on table-based fact verification, but a new trend is extending the scope to structured evidence. |
| Approach: | They propose a mixture-of-experts neural network to recognize and execute different types of reasoning . they use a management module to decide the contribution of each expert network to the verification result . |
| Outcome: | The proposed method achieves 85.1% accuracy on the TabFact dataset, comparable with the previous state-of-the-art models. |
Extraction of Message Sequence Charts from Software Use-Case Descriptions (N19-2)
Copied to clipboard
Girish Palshikar, Nitin Ramrakhiyani, Sangameshwar Patil, Sachin Pawar, Swapnil Hingmire, Vasudeva Varma, Pushpak Bhattacharyya
| Challenge: | Software Requirement Specification documents provide natural language descriptions of the core functional requirements as a set of use-cases. |
| Approach: | They propose a linguistic knowledge-based approach to extract software requirements from use-cases using a textual representation of the core functional requirements. |
| Outcome: | The proposed method performs better than existing techniques and improves performance. |
Verifying the Steps of Deductive Reasoning Chains (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models have been shown to improve the reasoning capabilities of the models. |
| Approach: | They propose to automate verification of individual reasoning steps in a logical deductive Chain-of-Thought. |
| Outcome: | The proposed method can detect unsound reasoning steps fairly well, but under-performs symbolic methods. |
MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Models (2025.emnlp-demos)
Copied to clipboard
Zhiwei Liu, Jielin Qiu, Shiyu Wang, Jianguo Zhang, Zuxin Liu, Roshan Ram, Haolin Chen, Weiran Yao, Shelby Heinecke, Silvio Savarese, Huan Wang, Caiming Xiong
| Challenge: | Existing evaluation frameworks suffer from limitations such as static task benchmarks, limited scope, and inadequate integration with practical applications. |
| Approach: | They propose an open-source, Model Context Protocol-based evaluation framework specifically tailored for comprehensive and systematic assessment of LLM-powered agents. |
| Outcome: | The proposed framework uncovers nuanced performance patterns and identify domain-specific strengths and weaknesses, providing valuable insights beyond traditional binary success metrics. |
ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models (2026.acl-demo)
Copied to clipboard
Jiasheng Zheng, Xin Zheng, Boxi Cao, Pengbo Wang, Zhengzhao Ma, Qiming Zhu, Jiazhen Jiang, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun
| Challenge: | Existing code sandboxes fail to provide accurate verification and efficiency under high-concurrency workloads. |
| Approach: | They propose a high-fidelity code verification system that provides sandbox feedback for RL training and evaluation. |
| Outcome: | The proposed system outperforms heuristic-matching baselines on LiveCodeBench and training stability on high-concurrency workloads. |
Claim Check-Worthiness Detection as Positive Unlabelled Learning (2020.findings-emnlp)
Copied to clipboard
| Challenge: | a unified approach to claim check-worthiness detection is a critical component of fact checking systems. |
| Approach: | They propose a unified approach which corrects for misinformation by positive unlabelled learning . they propose citation needed detection from Wikipedia and a ranking task which is a critical component of automatic fact checking systems. |
| Outcome: | The proposed method outperforms the state of the art in two of the three tasks studied in English. |
Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence (2021.naacl-main)
Copied to clipboard
| Challenge: | Typical fact verification models use retrieved written evidence to verify claims . evidence sources change over time as more information is gathered and revised . a new benchmark for fact verification is VitaminC, which is contrastive in nature . |
| Approach: | They propose a benchmark that uses Wikipedia revisions to train models to discern and adjust to slight factual changes. |
| Outcome: | The proposed model improves accuracy by 10% on adversarial fact verification and 6% on adversary natural language inference. |
CodeContests-O: Powering LLMs via Feedback-Driven Iterative Test Case Generation (2026.findings-acl)
Copied to clipboard
Jianfeng Cai, Jinhua Zhu, Ruopei Sun, Kangwen Zhao, Dongyun Xue, Mingxiao Feng, Wengang Zhou, Houqiang Li
| Challenge: | Existing approaches to synthesize test cases using Large Language Models (LLMs) rely on the model’s intrinsic generation capabilities without external feedback, resulting in insufficiently diverse cases. |
| Approach: | They propose a feedback-driven iterative framework that leverages Large Language Models to generate initial test cases, execute them against known correct and incorrect solutions, and utilizes the failed results as feedback to guide the LLM in refining the test cases toward high fidelity and discriminability. |
| Outcome: | The proposed method outperforms the existing codecontests and codecontests+ models by 4.30% and 8.78%. |
JointCQ: Improving Factual Hallucination Detection with Joint Claim and Query Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for detecting factual hallucinations in generated content exhibit limitations in the first two stages of the halluciation detection pipeline. |
| Approach: | They propose a joint claim-and-query generation framework that can detect factual hallucinations in generated content. |
| Outcome: | The proposed method outperforms existing methods on open-domain QA hallucination detection benchmarks. |
ProoFVer: Natural Logic Theorem Proving for Fact Verification (2022.tacl-1)
Copied to clipboard
| Challenge: | Recent fact verification systems rely on neural network classifiers for veracity prediction, which lack explainability. |
| Approach: | They propose a model that generates natural logic-based inferences as proofs using lexical mutations between spans in the claim and the evidence retrieved. |
| Outcome: | The proposed model has highest label accuracy and second best score in the FEVER leaderboard. |
NP-Hard Lower Bound Complexity for Semantic Self-Verification (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing work on semantic self-verification has been focused on mechanistic interpretation, but the computational complexity of semantic self verification has remained undercharacterized. |
| Approach: | They propose to formalize Semantic Self-Verification as problem of determining whether a statement accurately characterizes its own semantic properties within a given interpretive framework. |
| Outcome: | The proposed model is NP-complete and has implications for safety and fairness approaches . a 3-SAT formula is used to model an instance of SSV involving ambiguous terms . |
Zero-shot Fact Verification by Claim Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification require large datasets, which can be expensive. |
| Approach: | They propose a framework for training a robust fact verification model by using automatically generated claims that can be supported, refuted, or unverifiable from evidence from Wikipedia. |
| Outcome: | The proposed framework reduces the demand for human-annotated training data and improves a model's F1 from 50% to 77%, equivalent in performance to 2K+ manually-curated examples. |
Verifying Claims About Metaphors with Large-Scale Automatic Metaphor Identification (2024.naacl-short)
Copied to clipboard
| Challenge: | Existing studies on metaphors have focused on a small number of examples, whereas few studies verify claims with large corpus. |
| Approach: | They propose to use a large corpus to verify existing claims about verb metaphors . they apply metaphor detection to sentences extracted from Common Crawl . |
| Outcome: | The proposed method identifies verb metaphors with lower concreteness, imageability, familiarity and more emotional and subjective sentences. |
FinReporting: An Agentic Workflow for Localized Reporting of Cross-Jurisdiction Financial Disclosure (2026.acl-demo)
Copied to clipboard
Fan Zhang, Mingzi Song, Rania Elbadry, Yankai Chen, Shaobo Wang, Yixi Zhou, Xunwen Zheng, Yueru He, Yuyang Dai, Georgi Nenkov Georgiev, Ayesha Gull, Muhammad Usman Safder, Fan Wu, Liyuan Meng, Fengxian Ji, Junning Zhao, Xueqing Peng, Jimin Huang, YU Chen, Xue Liu, Preslav Nakov, Zhuohan Xie
| Challenge: | FinReporting is an agentic workflow for localized cross-jurisdiction financial reporting . existing approaches assume a single-market setting and overlook structural differences across jurisdictions . |
| Approach: | They propose a workflow that decomposes financial reporting into auditable stages . they use Large Language Models to extract and summarize corporate disclosures . |
| Outcome: | The proposed system decomposes reporting into auditable stages . it improves consistency and reliability under heterogeneous reporting regimes. |
AMenDeD: Modelling Concepts by Aligning Mentions, Definitions and Decontextualised Embeddings (2024.lrec-main)
Copied to clipboard
| Challenge: | Contextualised Language Models (LMs) improve on word embeddings by encoding meaning of words in context. |
| Approach: | They propose to learn a unified embedding space in which all three types of representations can be integrated. |
| Outcome: | The proposed model outperforms existing approaches in ontology completion tasks. |
FEVER: a Large-scale Dataset for Fact Extraction and VERification (N18-1)
Copied to clipboard
| Challenge: | 185,445 claims generated by altering sentences from Wikipedia are verified without knowledge of the sentence they were derived from. |
| Approach: | They propose a publicly available dataset for verification against textual sources, FEVER: Fact Extraction and VERification. |
| Outcome: | The proposed dataset achieves 31.87% accuracy on labeling a claim accompanied by the correct evidence, compared to 50.91% if we ignore the evidence. |
GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification (P19-1)
Copied to clipboard
| Challenge: | Existing methods to extract information from evidence are unable to grasp relational and logical information among the evidence. |
| Approach: | They propose a graph-based evidence aggregating and reasoning framework to integrate evidence from multiple pieces of evidence. |
| Outcome: | The proposed framework achieves significant performance improvements on a large-scale benchmark dataset. |
X-Fact: A New Benchmark Dataset for Multilingual Fact Checking (2021.acl-short)
Copied to clipboard
| Challenge: | Several fact-checking initiatives, such as PolitiFact, expend manual labor to investigate and determine the truthfulness of viral statements. |
| Approach: | They propose a multilingual dataset for factual verification of naturally existing claims . they use a benchmark to evaluate the multilingual models . |
| Outcome: | The proposed model achieves an F-score of around 40%, suggesting it is a challenging benchmark for multilingual fact-checking models. |
When Misinformation Speaks and Converses: Rethinking Fact-Checking in Audio Platforms (2026.acl-long)
Copied to clipboard
| Challenge: | Existing fact-checking pipelines focus on written claims, not on audio . authors argue that audio misinformation is structurally different because it is both spoken and conversational . |
| Approach: | They argue that audio misinformation is structurally different because it is both spoken and conversational . they argue that advancing fact-checking requires rethinking verification pipelines around spoken and conversations . |
| Outcome: | The proposed method fails on audio because it is both spoken and conversational . podcasts exceed 4.3 million distinct shows, reaching an estimated 500 million listeners globally . |
SemViQA: A Semantic Question Answering System for Vietnamese Information Fact-Checking (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing methods struggle with semantic ambiguity, homonyms, and complex linguistic structures, often trading accuracy for efficiency. |
| Approach: | They propose a Vietnamese fact-checking framework that integrates SER and TVC to achieve 78.97% strict accuracy. |
| Outcome: | The proposed framework achieves state-of-the-art accuracy with 78.97% strict accuracy on ISE-DSC01 and 80.82% on ViWikiFC while maintaining competitive accuracy. |
Heuristic Authorship Obfuscation (P19-1)
Copied to clipboard
| Challenge: | Existing methods for authorship verification are insufficient to control the authorial style of a text. |
| Approach: | They propose a novel method that models writing style difference as the Jensen-Shannon distance between character n-gram distributions of texts and manipulates an author’s subconsciously encoded writing style using heuristic search. |
| Outcome: | The proposed approach defeats state-of-the-art verification approaches while keeping text changes at a minimum. |
GuideTree: Guideline-Induced Review Trees for Long Medical Records (2026.acl-industry)
Copied to clipboard
| Challenge: | Medical record reviewers must produce consistent, traceable, guideline-compliant outcomes . longcontext inference is expensive and often degrades as inputs grow . |
| Approach: | a new method compiles textual guidelines into a fixed review tree . a cost-aware split-and-prune search is used to update the tree offline . the algorithm produces consistent, traceable, guideline-compliant outcomes . |
| Outcome: | The proposed system outperforms the strongest non-expert baselines by 84.5–92.8 Macro-F1 . it reduces average I/O volume to 74K input+output characters and average latency to 22s . |
Coordinating Search-Informed Reasoning and Reasoning-Guided Search in Claim Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for multi-hop claim verification require multi-step reasoning to construct verification chains while iterating for information to uncover hidden bridging facts. |
| Approach: | They propose a hierarchical agent reasoning and information search model that integrates reasoning and search-informed reasoning. |
| Outcome: | Experimental results show that HARIS improves multi-hop claim verification accuracy and interpretability. |
Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference (2025.findings-emnlp)
Copied to clipboard
Hao Mark Chen, Wayne Luk, Yiu Ka Fai Cedric, Rui Li, Konstantin Mishchenko, Stylianos Venieris, Hongxiang Fan
| Challenge: | Auto-regressive decoding of Large Language Models results in significant overheads in hardware performance . a novel parallel prompt decoding approach is proposed to overcome these limitations . |
| Approach: | They propose a parallel prompt decoding that uses a single model for speculation and verification. |
| Outcome: | The proposed approach speeds up auto-regressive decoding of large language models 2.49 times . it can be used on mobileLlama to Vicuna-13B on a wide range of benchmarks . |
Table-based Fact Verification with Self-labeled Keypoint Alignment (2022.coling-1)
Copied to clipboard
| Challenge: | Existing methods for fact verification rely on graph feature or data augmentation but fail to investigate evidence correlation between statement and table effectively. |
| Approach: | They propose a self-labeled keypoint alignment model to explore correlation between statement and table . they propose integrating a mixture-of experts block to integrate interacted information . |
| Outcome: | The proposed model outperforms the state-of-the-art models and captures interpretable evidence words on three widely-studied datasets. |
IMCI: Integrate Multi-view Contextual Information for Fact Extraction and Verification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing models for fact extraction and verification fail to utilize multi-view contextual information. |
| Approach: | They propose to integrate multi-view contextual information (IMCI) for fact extraction and verification by combining contextual information with inter-document context. |
| Outcome: | The proposed framework achieves state-of-the-art performance on the open-domain Wikipedia task with a winning FEVER score of 73.96% and label accuracy of 77.25% on the online blind test set. |
What Makes Medical Claims (Un)Verifiable? Analyzing Entity and Relation Properties for Fact Verification (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing studies show that identifying verifiable claims is difficult, whereas identifying unverifiably claims is more challenging. |
| Approach: | They hypothesize that breaking down claims into smaller units increases our understanding which properties impact verifiability. |
| Outcome: | The proposed corpus of evidence is based on the first corpus for scientific fact verification annotated with subject–relation–object triplets, evidence documents, and fact-checking verdicts. |
Self-Correcting Code Generation Using Small Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has demonstrated that self-correction is a powerful tool for code generation, but whether it is effective for smaller models remains unexplored. |
| Approach: | They propose a method that trains small language models to maintain correct outputs while progressively correcting incorrect outputs as turns proceed. |
| Outcome: | The proposed approach improves the ability of small language models for multi-turn code correction. |
Topic-Aware Evidence Reasoning and Stance-Aware Aggregation for Fact Verification (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fact verification focus on analyzing semantic interaction between claim and evidence but fail to capture their topical consistency . Existing models focus on the aggregation of multiple pieces of evidence without considering their implicit stances to the claim, thereby introducing spurious information. |
| Approach: | They propose a topic-aware evidence reasoning and stance-again aggregation model that checks topical consistency between claims and evidence. |
| Outcome: | The proposed model outperforms state-of-the-art models on two benchmark datasets. |
Calibration of Machine Reading Systems at Scale (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to calibrate open setting machine reading systems fail to scale to these settings due to various scale limitations in practical settings. |
| Approach: | They propose to extend existing calibration approaches to calibrate open-domain question answering and claim verification systems to these settings. |
| Outcome: | The proposed calibration methods can selectively predict answers when question answering systems are posed with unanswerable or out-of-the-training distribution questions. |
Table-Text Alignment: Explaining Claim Verification Against Tables in Scientific Papers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | predicting the final label alone is insufficient and offers limited interpretability. |
| Approach: | They propose to reframe table–text alignment as an explanation task requiring models to identify the table cells essential for claim verification. |
| Outcome: | The proposed taxonomy improves claim verification performance and most LLMs fail to recover human-aligned rationales, suggesting that their predictions do not stem from faithful reasoning. |
TEN: Table Explicitization, Neurosymbolically (2026.acl-industry)
Copied to clipboard
| Challenge: | Existing methods for extracting tabular data from semistructured text are error-prone and costly. |
| Approach: | They propose a neurosymbolic approach to extract tabular data from semistructured text . TEN is a triadic feedback loop that iteratively refines table hypotheses . |
| Outcome: | The proposed approach outperforms neural baselines in exact match accuracy and lower hallucination rates. |
GLAF: Global-to-Local Aggregation and Fission Network for Semantic Level Fact Verification (2022.coling-1)
Copied to clipboard
| Challenge: | Existing fact verification models lack fine-grained reasoning over key entities . GLAF uses local fission reasoning to capture latent logical relations between clues . |
| Approach: | They propose a global-to-local fission and fissional network to capture latent logical relations hidden in multiple evidence clues. |
| Outcome: | The proposed network achieves state-of-the-art on a FEVER dataset with a 77.62% FEVER score. |
WaterSeeker: Pioneering Efficient Detection of Watermarked Segments in Large Documents (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods focus on distinguishing fully watermarked text from non-watermarked text, overlooking real-world scenarios where LLMs generate only brief segments within longer documents. |
| Approach: | They propose a method to detect watermarked segments in large documents using an anomaly extraction method and a local traversal. |
| Outcome: | The proposed method achieves a superior balance between detection accuracy and computational efficiency. |
Towards Reward Fairness in RLHF: From a Resource Allocation Perspective (2025.acl-long)
Copied to clipboard
| Challenge: | if rewards are imperfect, they can adversely affect the alignment of large language models (LLMs). |
| Approach: | They propose a bias-agnostic method to address the issue of reward unfairness from a resource allocation perspective without specifically designing for each type of bias . they apply methods Fairness Regularization and Fairness Coefficient to achieve fairness in rewards. |
| Outcome: | The proposed method achieves fairness in rewards while minimizing biases . it can be applied to verification and reinforcement learning scenarios . |
Recon, Answer, Verify: Agents in Search of Truth (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing benchmark datasets suffer from leakage or evidence incompleteness, limiting the realism of current evaluations. |
| Approach: | They propose an agentic framework that iteratively generates and answers sub-questions to verify different aspects of the claim before finally generating the label. |
| Outcome: | The proposed system outperforms existing methods by 57.5% on Politi-Fact-Only and 3.05% on the widely used HOVER datasets. |
LongFaith: Enhancing Long-Context Reasoning in LLMs with Faithful Synthetic Data (2025.findings-acl)
Copied to clipboard
| Challenge: | Long-context processing ability has emerged as a significant challenge for large language models. |
| Approach: | They propose a pipeline for synthesizing faithful long-context reasoning instruction datasets . they integrate ground truth and citation-based reasoning prompts integrating them . |
| Outcome: | The proposed pipeline eliminates distractions and improves reasoning chains. |
SYNTHVERIFY: Enhancing Zero-Shot Claim Verification through Step-by-Step Synthetic Data Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for claim verification are inefficient or rely on external documents. |
| Approach: | They propose a step-by-step prompting-based synthetic data generation framework to enhance zero-shot claim verification. |
| Outcome: | The proposed framework bridges LLMs’ knowledge gaps in specialized domains without access to external corpora or sacrificing generalizability. |
Verification and Refinement of Natural Language Explanations through LLM-Symbolic Theorem Proving (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for assessing the validity of explanations for NLI are time-consuming and prone to logical errors. |
| Approach: | They propose a framework that integrates Large Language Models and Theorem Provers to verify and refine natural language explanations through crowd-sourcing . they propose to use TPs to generate and formalise explanatory sentences and suggest potential inference strategies for NLI. |
| Outcome: | The proposed framework generates and formalises explanatory sentences and suggests potential inference strategies for NLI. |
Learning Invariant Representations of Social Media Users (D19-1)
Copied to clipboard
| Challenge: | Existing methods for learning to compare social media users fail to generalize to new users or even to previously known users. |
| Approach: | They propose a procedure to learn a mapping from short episodes of user activity to a vector space in which the distance between points captures the similarity of the corresponding users’ invariant features. |
| Outcome: | The proposed procedure can be applied to users not seen at training time and enables efficient comparisons of users in the resulting vector space. |
Attribute First, then Generate: Locally-attributable Grounded Text Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Recent efforts to address hallucinations in Large Language Models have focused on attributed text generation, which supplements generated texts with citations of supporting sources for post-generation fact-checking and corrections. |
| Approach: | They propose a locally-attributable text generation approach prioritizing concise attributions by identifying relevant source segments and conditioning the generation process on them. |
| Outcome: | The proposed method yields more concise citations than baselines and significantly reduces time required for fact verification by human assessors. |
TABED: Test-Time Adaptive Ensemble Drafting for Robust Speculative Decoding in LVLMs (2026.findings-eacl)
Copied to clipboard
Minjae Lee, Wonjun Kang, Byeongkeun Ahn, Christian Classen, Kevin Galim, Seunghyuk Oh, Minghao Yan, Hyung Il Koo, Kangwook Lee
| Challenge: | Large Vision Language Models (LVLMs) are advanced models that process multiple modalities, such as images, audio, and video, alongside text. |
| Approach: | They propose to use a method to generate and verify draft tokens in parallel . they compare existing methods with small draft models and observe performance fluctuations . |
| Outcome: | The proposed method achieves an average walltime speedup of 1.74 over autoregressive decoding and a 5% improvement over single drafting methods. |
Adapting Open Domain Fact Extraction and Verification to COVID-FACT through In-Domain Language Modeling (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to verify scientifically false online information are limited by the lack of training data in the scientific domain. |
| Approach: | They propose an in-domain language modeling method for fact extraction and verification systems . they use SCIFACT to extract scientifically false online information . |
| Outcome: | The proposed method improves accuracy 30% on SCIFACT dataset . state-of-the-art model achieves only 46.6% precision, which is hard to be trusted for users. |
QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on fact verification focuses on news, tables and Wikipedia passages. |
| Approach: | They propose a question-answering dialogue based fact verification with mixture of experts that exploits questions and evidence effectively in the verification process. |
| Outcome: | The proposed approach outperforms previous approaches on three benchmark datasets and achieves state-of-the-art results. |
INSURE-Dial: A Phase-Aware Conversational Dataset Benchmark for Compliance Verification and Phase Detection (2026.eacl-long)
Copied to clipboard
| Challenge: | Administrative phone tasks drain roughly $1 trillion annually from healthcare, with over 500 million calls manually handled in 2024. |
| Approach: | They propose a new benchmark for developing and assessing compliance-aware voice agents for phase-against-span-based call auditing with span-based compliance verification. |
| Outcome: | The proposed framework is the first public benchmark for developing and assessing compliance-aware voice agents for phase-against-span-based call auditing with span-based compliance verification. |
Constrained Decoding with Speculative Lookaheads (2025.naacl-long)
Copied to clipboard
Nishanth Sridhar Nakshatri, Shamik Roy, Rajarshi Das, Suthee Chaidaroon, Leonid Boytsov, Rashmi Gangadharaiah
| Challenge: | Constrained decoding with lookahead heuristics is effective for aligning LLM generations to human preferences, but the extensive lookaheaded roll-out operations for each generated token make it prohibitively expensive. |
| Approach: | They propose a technique that uses lookaheads to align LLMs to human preferences . they propose 2.2x to 12.15x speedup over greedy decoding . |
| Outcome: | The proposed technique achieves 2.2x to 12.15x speedup over greedy decoding without significant performance reduction. |
MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrieval (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing methods for verification of claims rely on textual evidence only or ignore the explainability. |
| Approach: | They propose a multi-modal reasoning model that integrates text and visual evidence for verification. |
| Outcome: | The proposed model achieves evidence retrieval, multi-modal claim verification, and explanation generation. |
GuideDog: A Real-World Egocentric Multimodal Dataset for Blind and Low-Vision Accessibility-Aware Guidance (2026.acl-long)
Copied to clipboard
Junhyeok Kim, Jaewoo Park, Junhee Park, Sangeyl Lee, Jiwan Chung, Jisung Kim, Ji Hoon Joung, Youngjae Yu
| Challenge: | Recent advances in multimodal large language models (MLLMs) offer new opportunities for higher-level scene understanding, but they require labor-intensive, expert annotation. |
| Approach: | They propose a dataset that combines 2K human-verified images with 22K image-description pairs to provide a more accurate representation of pedestrian scenes. |
| Outcome: | The proposed dataset improves scalability while maintaining quality. |
Optimizing Decomposition for Optimal Claim Verification (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decomposition and verification paradigms ignore their interactions and potential misalignment. |
| Approach: | They propose a reinforcement learning framework that leverages verifier feedback to learn a policy for dynamically decomposing claims to verifier-preferred atomicity. |
| Outcome: | The proposed framework outperforms existing decomposition policies in verification confidence tests . it improves accuracy and confidence by 0.12 on average across varying verifiers, datasets, and atomcities of input claims. |
Evidence-based Factual Error Correction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to correct factual errors are limited to labeled claims . a recent task of fact verification has attracted significant attention . |
| Approach: | They propose a task of factual error correction that performs edits to a claim so that the generated rewrite is better supported by evidence. |
| Outcome: | The proposed method produces accurate factual error corrections for 5x more instances in human evaluation and a .125 increase in SARI score. |
TextVerifier: Robustness Verification for Textual Classifiers with Certifiable Guarantees (2023.findings-acl)
Copied to clipboard
| Challenge: | a textual classifier must withstand word-level alteration attacks due to inherent vulnerability. |
| Approach: | They propose a formal verification framework with certifiable guarantees on deep neural networks in natural language processing against word-level alteration attacks. |
| Outcome: | The proposed framework provides an approximation of the maximal safe radius with tight bounds . it yields an efficient speed edge and reliable anytime estimation . |
Select-Then-Decompose: From Empirical Analysis to Adaptive Selection Strategy for Task Decomposition in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing task decomposition methods focus on memory, tool usage, and feedback mechanisms, but they often overlook the trade-off between performance and cost. |
| Approach: | They propose a strategy that selects the most suitable decomposition approach based on task characteristics and enhances the reliability of the results through a verification module. |
| Outcome: | The proposed strategy is based on categories of approaches, characteristics of tasks, and configuration of decomposition and execution models. |
ClaimDiff: Comparing and Contrasting Claims on Contentious Issues (2023.findings-acl)
Copied to clipboard
| Challenge: | Using fact verification tasks, however, can not detect subtle differences in factually consistent claims, which might bias the readers. |
| Approach: | They propose a novel dataset that primarily focuses on comparing the nuance between claim pairs. |
| Outcome: | The proposed dataset shows that human-labeled 2,941 claim pairs are weaker than baselines, showing a 19% absolute gap with the baselines. |
Evaluating adversarial attacks against multiple fact verification systems (D19-1)
Copied to clipboard
| Challenge: | Automated fact verification is progressing due to advances in modeling and availability of large datasets. |
| Approach: | They propose two scoring metrics which take into account the correctness of adversarial instances. |
| Outcome: | The proposed method and paraphrasing method have higher potency and higher resilience than baselines. |
Benchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing QA-based summarization metrics must automatically determine whether the QA model’s prediction is correct or not. |
| Approach: | They benchmark lexical answer verification methods used by current QA-based metrics and two more sophisticated text comparison methods, BERTScore and LERC. |
| Outcome: | The proposed methods outperform the other methods in some settings while remaining statistically indistinguishable from lexical overlap in others. |
QuantileMark: A Message-Symmetric Multi-bit Watermark for LLMs (2026.acl-long)
Copied to clipboard
| Challenge: | a number of large language models (LLMs) require multi-bit watermarking to ensure provenance. |
| Approach: | They propose a multi-bit watermark that embeds messages within a continuous cumulative probability interval. |
| Outcome: | The proposed watermark breaks message symmetry in low-entropy decoding, showing it can be used for verification and quality verification. |
E-Verify: A Paradigm Shift to Scalable Embedding-based Factuality Verification (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing factuality verification methods follow a Decompose-Then-Verify paradigm, which improves granularity but suffers from poor scalability and efficiency. |
| Approach: | They propose a Decompose-Embed-Interact paradigm that shifts factuality verification from costly text-level reasoning to efficient alignment in embedding space. |
| Outcome: | The proposed paradigm shifts factuality verification from costly text-level reasoning to efficient alignment in embedding space . |
Distill and Align Decomposition for Enhanced Claim Verification (2026.findings-eacl)
Copied to clipboard
Jabez Magomere, Elena Kochkina, Samuel Mensah, Simerjot Kaur, Fernando Acero, Arturo Oncevay, Charese Smiley, Xiaomo Liu, Manuela Veloso
| Challenge: | Existing methods for complex claim verification struggle to align decomposition quality with verification performance. |
| Approach: | They propose a reinforcement learning approach that optimizes decomposition quality and verifier alignment using Group Relative Policy Optimization. |
| Outcome: | The proposed method outperforms prompt-based approaches and existing methods in six evaluation settings. |
Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing autoregressive models generate tokens sequentially and are memory-bound, resulting in a memory-based inference stage that is memory-limited. |
| Approach: | They propose an approach to accelerate the inference speed of large language models with billions of parameters by integrating semi-autoregressive inference and speculative decoding capabilities. |
| Outcome: | The proposed approach has demonstrated inference speedups of 2.7x-4.0x on humanEval-X while maintaining output quality. |
MCiteBench: A Multimodal Benchmark for Generating Text with Citations (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work focuses on generating citations for text-only content . experimental results reveal MLLMs struggle to ground outputs reliably when handling multimodal input . |
| Approach: | They propose a benchmark to assess the ability of MLLMs to generate text with citations in multimodal contexts. |
| Outcome: | The proposed benchmark assesses the ability of MLLMs to generate text with citations in multimodal contexts. |
Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance? (2025.naacl-long)
Copied to clipboard
| Challenge: | Fact-checking pipelines adopt the Decompose-Then-Verify paradigm, where texts are broken down into smaller claims for individual verification and subsequently combined for a veracity decision. |
| Approach: | They propose to categorize decomposition errors and to reveal a trade-off between accuracy gains and noise introduced by decomposing. |
| Outcome: | The proposed analysis provides new insights into understanding current system’s instability and offers guidance for future studies toward improving claim decomposition in fact-checking pipelines. |
RASD: Retrieval-Augmented Speculative Decoding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for generating draft tokens rely on lightweight draft models or additional model structures to generate tokens and retrieve context from databases. |
| Approach: | They propose to use a pruning method to enhance model-based speculative decoding by combining the best-fit model with the best retrieval tree. |
| Outcome: | The proposed method achieves state-of-the-art inference acceleration across tasks such as DocQA, Summary, Code, and In-Domain QA. |
What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in reasoning with large language models have popularized Long Chain-of-Thought (LCoT) a framework that converts sequential LCoTs into hierarchical tree structures enables deeper structural analysis of LLM reasoning. |
| Approach: | They propose a framework that converts sequential LCoTs into hierarchical tree structures and enables deeper structural analysis of LLM reasoning. |
| Outcome: | The proposed framework can be used to analyze LLM reasoning in a variety of tasks and models. |
PASTA: Table-Operations Aware Fact Verification via Sentence-Table Cloze Pre-training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Table-based fact verification has attracted a lot of attention recently due to the lack of datasets that can be used to pre-train language models to be aware of common table operations. |
| Approach: | They propose a table-based fact verification tool that pre-trains language models to be aware of common table operations such as aggregating a column or comparing tuples. |
| Outcome: | The proposed method outperforms previous methods on two table-based fact verification datasets TabFact and SEM-TAB- FACTS. |
TrigReason: Trigger-Based Collaboration between Small and Large Reasoning Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Reasoning Models suffer from high inference latency due to autoregressive reasoning . SpecReason adopts a polling-based design that repeatedly invokes the LRM for verification at every step . |
| Approach: | They propose a trigger-based collaborative reasoning framework that delegates most reasoning to the SRM and activates LRM intervention only when necessary. |
| Outcome: | The proposed framework reduces latency and API cost by 73.3% under edge–cloud conditions. |
User-Centric Evidence Ranking for Attribution and Fact Verification (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models often present users with insufficient or redundant information, leading to inefficient and error-prone verification. |
| Approach: | They propose a task that prioritizes presenting sufficient information as early as possible in a ranked list. |
| Outcome: | The proposed task minimizes user reading effort while making all available evidence accessible for sequential verification. |
Towards Debiasing Fact Verification Models (D19-1)
Copied to clipboard
Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, Regina Barzilay
| Challenge: | Prior research has shown that data collection methods that use crowdsourcing introduce idiosyncratic biases that impact performance in unexpected ways. |
| Approach: | They propose a method to regularize the training data to avoid idiosyncrasies in the datasets that are used for fact verification. |
| Outcome: | The proposed model outperforms the existing model on the FEVER dataset, achieving 61.7% of the baseline. |
FactAppeal: Identifying Epistemic Factual Appeals in News Media (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods focus on the content of factual statements and ignore the epistemic structures that confer credibility and persuasive force to these claims. |
| Approach: | They propose a task of Epistemic Appeal Identification to identify whether and how factual statements have been anchored by external sources or evidence. |
| Outcome: | The proposed task identifies whether and how factual statements have been anchored by external sources or evidence. |
SciFact-Open: Towards open-domain scientific claim verification (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Current scientific claim verification systems can achieve very strong performance on limited contexts, in some cases approaching human agreement. |
| Approach: | They propose to pool and annotate top predictions from four state-of-the-art scientific claim verification models to evaluate their performance against large corpora. |
| Outcome: | The proposed system performs well on a corpus of 500K scientific abstracts. |
Beyond Facts: Evaluating Intent Hallucination in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) produce unsatisfactory results when faced with complex queries containing multiple conditions. |
| Approach: | They propose a benchmark for intent hallucination that covers 20,068 problems and an automatic LLM generation evaluation metric for detecting intent hallucinosis. |
| Outcome: | The proposed benchmark covers query-only and retrieval-augmented generation (RAG) setups with varying topics and difficulty. |
Verification-Aware Planning for Multi-Agent Systems (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language model (LLM) agents are increasingly deployed to tackle complex tasks . multi-agent collaboration introduces new challenges in planning, coordination, and verification . |
| Approach: | They propose a framework for multi-agent collaboration with verification-aware planning . the framework decomposes tasks, models subtask dependencies, and encodes planner-defined passing criteria as subtask verification functions (VFs) |
| Outcome: | The proposed framework outperforms baselines on diverse datasets while improving system robustness and interpretability. |
FaVIQ: FAct Verification from Information-seeking Questions (2022.acl-long)
Copied to clipboard
| Challenge: | Existing fact verification datasets with crowdsourced claims introduce subtle biases that are difficult to control for. |
| Approach: | They construct a large-scale fact verification dataset with ambiguous questions . they use a corpus of 188k claims to construct false and true claims . |
| Outcome: | The proposed dataset outperforms models trained on the dataset FEVER or in-domain data by up to 17% absolute. |
Counterfactual Debiasing for Fact Verification (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods for debiasing factchecking models learn such biases instead of understanding the semantic relationship between the claim and evidence. |
| Approach: | They propose a counterfactual framework CLEVER which is augmentation-free and mitigates biases on the inference stage. |
| Outcome: | The proposed method is augmentation-free and mitigates biases on the inference stage. |
Instructing Large Language Models to Identify and Ignore Irrelevant Conditions (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing CoT prompting methods elicited multi-step reasoning abilities of large language models (LLMs) but they were seriously confused by the irrelevant conditions, resulting in low accuracy. |
| Approach: | They propose a method that instructs large language models to identify and ignore irrelevant conditions and prompts them to verify the irrelevant conditions. |
| Outcome: | The proposed approach outperforms existing methods on MWPs with GPT-3.5-Turbo and I3C-Select. |
HoneyBee: Progressive Instruction Finetuning of Large Language Models for Materials Science (2023.findings-emnlp)
Copied to clipboard
| Challenge: | LLaMa-based language model for materials science is first of its kind in the world . |
| Approach: | They propose an instruction-based process for trustworthy data curation in materials science (MatSci-Instruct) they then apply this process to finetune a LLaMa-based language model targeted for materials science. |
| Outcome: | The proposed model outperforms existing language models on materials science tasks and improves in successive stages of refinement. |
Forward-Backward Reasoning in Large Language Models for Mathematical Verification (2024.findings-acl)
Copied to clipboard
| Challenge: | Extensive experiments on six standard mathematical data sets and three LLMs show that FOBAR achieves state-of-the-art performance. |
| Approach: | They propose to combine forward and backward reasoning to verify candidate answers . they propose to use a template to mask a number and ask the LLM to answer a backward question . |
| Outcome: | Experiments on mathematical data show that proposed backward reasoning outperforms Self-Consistency. |
NewsClaims: A New Benchmark for Claim Detection from News with Attribute Knowledge (2022.emnlp-main)
Copied to clipboard
Revanth Gangi Reddy, Sai Chetan Chinthakindi, Zhenhailong Wang, Yi Fung, Kathryn Conger, Ahmed ELsayed, Martha Palmer, Preslav Nakov, Eduard Hovy, Kevin Small, Heng Ji
| Challenge: | Current claims detection methods focus on sentence analysis, ignoring other attributes . a key element of identifying misinformation is detecting the claims and the arguments that have been presented. |
| Approach: | They propose a benchmark for attribute-aware claim detection in the news domain . they extend the problem to include extraction of additional attributes related to each claim . |
| Outcome: | The proposed system performs well on the test, but human performance is still poor. |
MARS: Unleashing the Power of Speculative Decoding via Margin-Aware Verification (2026.findings-acl)
Copied to clipboard
Jingwei Song, Xinyu Wang, Hanbin Wang, Xiaoxuan Lei, Tianyu Shi, Shixin Han, Eric Yang, Xiao-Wen Chang, Lynn Ai
| Challenge: | Autoregressive large language models suffer from high inference latency due to memorybandwidth constraints. |
| Approach: | They propose a method that decouples generation and verification by decoupling tokens and a lightweight draft model. |
| Outcome: | The proposed method delivers consistent and significant speedups over state-of-the-art baselines while preserving generation quality across diverse benchmarks. |
Evidence Retrieval for Fact Verification using Multi-stage Reranking (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evidence retrieval methods are limited by single-stage evidence extraction. |
| Approach: | They propose to use a multi-stage reranking paradigm to enhance the fact verification process by increasing the recall of sentences by 7.85%, tables by 8.29% and cells by 3% compared to the current state-of-the-art. |
| Outcome: | The proposed system outperforms state-of-the-art models and achieves a 93.63% recall rate for Wikipedia pages. |
Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervision (2024.findings-emnlp)
Copied to clipboard
| Challenge: | a method for process supervision has shown significant improvements in multi-step problem solving . despite the advances in process supervision, there are still easily observable mistakes in state-of-the-art LLMs. |
| Approach: | They propose a method for automating data curation by using a trained verifier to evaluate intermediate steps generated by a reasoner. |
| Outcome: | The proposed method improves the performance of PaLM 2 on math and coding tasks. |
From Chaos to Clarity: Claim Normalization to Empower Fact-Checking (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Social media posts are noisy and pervasive, resulting in difficult to identify precise and prominent claims that require verification. |
| Approach: | They propose a task called Claim Normalization that decomposes complex and noisy social media posts into more straightforward and understandable forms, termed normalized claims. |
| Outcome: | The proposed model outperforms baselines across evaluation measures and errors. |
GuardEmb: Dynamic Watermark for Safeguarding Large Language Model Embedding Service Against Model Stealing Attack (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies reveal the risk of the model stealing attack, posing a financial threat to EaaS providers. |
| Approach: | They propose a dynamic embedding watermarking method that detects watermarks in embedded text . this method is a cross-platform approach that trains a verifier to detect watermark . |
| Outcome: | The proposed method enables an attacker to replicate the proposed method for profit without compromising embedding functionality. |
MIND: Multimodal Shopping Intention Distillation from Large Vision-language Models for E-commerce Purchase Understanding (2024.emnlp-main)
Copied to clipboard
Baixuan Xu, Weiqi Wang, Haochen Shi, Wenxuan Ding, Huihao Jing, Tianqing Fang, Jiaxin Bai, Xin Liu, Changlong Yu, Zheng Li, Chen Luo, Qingyu Yin, Bing Yin, Long Chen, Yangqiu Song
| Challenge: | Existing methods for acquiring large-scale intentions generate product-centric intentions without product images and incur high costs for scalability. |
| Approach: | They propose a multimodal framework that allows Large Vision-Language Models to infer purchase intentions from multimodal product metadata and prioritize human-centric ones. |
| Outcome: | The proposed framework shows that it is robust to different prompts and superior to previous methods. |
WiCE: Real-World Entailment for Claims in Wikipedia (2023.emnlp-main)
Copied to clipboard
| Challenge: | Textual entailment models are increasingly used in fact-checking, presupposition verification in question answering, or summary evaluation. |
| Approach: | They propose a new fine-grained textual entailment dataset built on natural claim and evidence pairs extracted from Wikipedia that provides en-tailment judgments over sub-sentence units of the claim and a minimal subset of evidence sentences that support each subclaim. |
| Outcome: | The proposed dataset improves on multiple datasets at test time and shows that real claims involve verification and retrieval problems that existing models fail to address. |
Stepwise Verification and Remediation of Student Reasoning Errors with Large Language Model Tutors (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for dialog tutoring fail to detect student errors and tailor their feedback to them. |
| Approach: | They propose to build dialog tutoring models to scaffold students' problem-solving and verify student solutions by using automatic and human evaluation. |
| Outcome: | The proposed model improves the quality of the tutor response generation by detecting student errors and adjusting the feedback to the errors. |
DuetSim: Building User Simulator with Dual Large Language Models for Task-Oriented Dialogues (2024.lrec-main)
Copied to clipboard
| Challenge: | User Simulators are used to train task-oriented dialogue systems . traditional training paradigms rely on human-engineered agendas resulting in generated responses that lack diversity and spontaneity. |
| Approach: | They propose a framework that leverages large language models to generate diverse responses . they use two LLMs to generate and verify responses, which are preferred by users . |
| Outcome: | The proposed framework produces responses that exhibit diversity and are preferred by human users. |
Reasoning-Based Refinement of Unsupervised Text Clusters with LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Unsupervised methods are used to induce latent semantic structure from large text collections, yet their outputs often contain incoherent, redundant, or poorly grounded clusters. |
| Approach: | They propose a framework that leverages large language models as semantic judges to validate and restructure unsupervised clustering algorithms. |
| Outcome: | The proposed framework improves cluster coherence and human-aligned labeling quality over traditional models and representation-based baselines. |
I am a Strange Dataset: Metalinguistic Tests for Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for metalinguistic self-reference are limited by the number of subtasks. |
| Approach: | They propose a dataset that aims to address metalinguistic self-reference in large language models. |
| Outcome: | The proposed dataset is hand-crafted by experts and validated by non-expert annotators. |
SCITAB: A Challenging Benchmark for Compositional Reasoning and Claim Verification on Scientific Tables (2023.emnlp-main)
Copied to clipboard
| Challenge: | Current scientific fact-checking benchmarks exhibit several shortcomings, such as biases arising from crowd-sourced claims and an over-reliance on text-based evidence. |
| Approach: | They present a dataset of 1.2K expert-verified scientific claims that require compositional reasoning for verification. |
| Outcome: | The proposed model outperforms existing models in table-based pretraining models and large language models. |
AutoGUI: Scaling GUI Grounding with Automatic Functionality Annotations from LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Existing datasets for UI-VLMs contain large-scale context-free element annotations or contextualized functional descriptions for elements at a small scale. |
| Approach: | They propose an auto-annotation pipeline that generates massive UI element functionality annotations based on UI content changes induced by interacting with the elements. |
| Outcome: | The proposed pipeline improves accuracy and scales well with human evaluation of a high-quality AutoGUI-704k dataset. |
Are LLM-based Evaluators Confusing NLG Quality Criteria? (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that LLMs confuse evaluation criteria, which reduces their reliability. |
| Approach: | They propose a hierarchical classification system for 11 common aspects with corresponding different evaluation criteria. |
| Outcome: | The proposed system is based on 11 common aspects with different evaluation criteria. |
The Confidence Dichotomy: Analyzing and Mitigating Miscalibration in Tool-Use Agents (2026.acl-long)
Copied to clipboard
| Challenge: | a fundamental pillar of trustworthiness is calibration, which refers to an agent’s ability to express confidence that reliably reflects its actual performance. |
| Approach: | They propose a reinforcement learning framework that jointly optimizes task accuracy and calibration, supported by a holistic benchmark of reward designs. |
| Outcome: | The proposed framework improves calibration across tool types and shows that trained agents achieve superior calibration and exhibit robust generalization from local training environments to noisy web settings and to distinct domains such as mathematical reasoning. |
QA-NatVer: Question Answering for Natural Logic-based Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work has focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim and its evidence via set-theoretic operators. |
| Approach: | They propose to use question answering to predict natural logic operators using generalization capabilities of instruction-tuned language models. |
| Outcome: | The proposed approach outperforms the best baseline on a Danish verification dataset by 4.3 accuracy points. |
IDTraffickers: An Authorship Attribution Dataset to link and connect Potential Human-Trafficking Operations on Text Escort Advertisements (2023.emnlp-main)
Copied to clipboard
| Challenge: | a significant number of human trafficking cases are associated with online advertisements . identification of HT vendors is challenging for law enforcement agencies . |
| Approach: | IDTraffickers uses 87,595 text ads and 5,244 vendor labels to link HT vendors . a macro-F1 score is achieved in a closed-set classification environment . |
| Outcome: | IDTraffickers is a dataset that enables verification and identification of HT vendors . the model achieves a macro-F1 score in a closed-set classification environment . |
Learning to Generate Programs for Table Fact Verification via Structure-Aware Semantic Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for table fact verification do not study generating latent programs from statements . current weakly supervised methods are limited due to huge search space with lots of spurious programs. |
| Approach: | They propose a structure-aware approach to do table fact verification by structure-based parsing . they leverage lexical features and structure features to generate program from statements . |
| Outcome: | The proposed method generates programs more accurately than existing parsers and achieves comparable performance to the SOTA on the large-scale benchmark TABFACT. |
Joint Verification and Reranking for Open Fact Checking Over Tables (2021.acl-long)
Copied to clipboard
Michael Sejr Schlichtkrull, Vladimir Karpukhin, Barlas Oguz, Mike Lewis, Wen-tau Yih, Sebastian Riedel
| Challenge: | Existing research into structured data has focused on textual data and the closed-domain setting is not reflective of real-world fact checking tasks. |
| Approach: | They propose a joint reranking-and-verification model which fuses evidence documents in the verification component and a heuristic retrieval baseline. |
| Outcome: | The proposed model achieves comparable performance to the closed-domain state-of-the-art on the TabFact dataset and significantly improves over a heuristic retrieval baseline. |
Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models (2026.acl-long)
Copied to clipboard
Jingwei Ni, Ekaterina Fadeeva, Tianyi Wu, Mubashara Akhtar, Jiaheng Zhang, Elliott Ash, Markus Leippold, Timothy Baldwin, See-Kiong Ng, Artem Shelmanov, Mrinmaya Sachan
| Challenge: | Existing verification approaches, such as Process Reward Models, are computationally expensive and limited to specific domains. |
| Approach: | They propose a transformer-based probe that uses internal states of frozen LLMs to estimate credibility of reasoning steps during generation. |
| Outcome: | The proposed probes match or exceed PRMs that are up to 810 larger. |
Compartmentalised Agentic Reasoning for Clinical NLI (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models produce fluent judgments for clinical natural language inference, yet fail when the decision requires the correct inferential schema rather than surface matching. |
| Approach: | They propose a compartmentalised agentic framework that routes each premise–statement pair to a reasoning family and applies a specialised solver with explicit verification and targeted refinement. |
| Outcome: | The proposed framework improves mean accuracy from 23% with direct prompting to 57%, with the largest gains on structurally demanding reasoning types. |
LiveFact: A Dynamic, Time-Aware Benchmark for LLM-Driven Fake News Detection (2026.acl-long)
Copied to clipboard
| Challenge: | Current evaluation frameworks are static and vulnerable to benchmark data contamination . current models are ineffective at assessing reasoning under temporal uncertainty . |
| Approach: | They propose a live-based benchmark that simulates the real-world "fog of war" they propose evaluating models on their ability to reason with evolving, incomplete information . |
| Outcome: | The proposed model outperforms proprietary state-of-the-art models in classification and evidence mode . it also provides a component to monitor BDC explicitly . |
LLM-SLM Collaborative Framework of Idiomatic Expression Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for idiomatic expression generation lack parallel data and manual annotations. |
| Approach: | They propose an iterative LLM-SLM collaborative framework that replaces human supervision for idiomatic expression data generation. |
| Outcome: | The proposed framework outperforms DeepSeek-R1 in Chinese Idiom Polishing with a 25.2% improvement in accuracy. |
Retrieval Augmented Fact Verification by Synthesizing Contrastive Arguments (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to verify claim credibility rely on embedded knowledge or unreliable context. |
| Approach: | They propose retrieval augmented fact verification through the synthesis of contrasting arguments (RAFTS) they use an embedding model to identify informative demonstrations and in-context prompts to generate the prediction and explanation. |
| Outcome: | The proposed method outperforms existing methods with smaller LLMs or unreliable contexts. |
EX-FEVER: A Dataset for Multi-hop Explainable Fact Verification (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on fact verification lack a high-quality dataset for explainability . existing systems lack evidence retrieval and veracity prediction, limiting the ability to verify a claim . |
| Approach: | They propose a dataset for multi-hop explainable fact verification that summarises and modifies Wikipedia documents. |
| Outcome: | The proposed dataset aims to improve the accuracy of multi-hop explainable fact verification systems. |
How Entangled is Factuality and Deception in German? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing research on deception detection and fact checking conflates factual accuracy with truthfulness . a belief-based deception framework defines texts as deceptive when there is a mismatch between what people say and what they truly believe . |
| Approach: | They assess if presumed patterns of deception generalize to German language texts . they gauge the impact of deceptiveness on the downstream task of fact checking . |
| Outcome: | The proposed framework disentangles deception when there is a mismatch between what people say and what they truly believe . the proposed framework does not find any correlation with established cues of deception . |
Evaluating Automatic Speech Recognition Systems for Korean Meteorological Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using a dataset of Korean weather queries, we find that automatic speech recognition systems fail on specialized vocabulary. |
| Approach: | They propose an evaluation dataset of Korean weather queries . the dataset was recorded by diverse native speakers following pronunciation guidelines . |
| Outcome: | The proposed model reduces error rates on meteorological terms and improves overall recognition accuracy. |
Evaluating Implicit Biases in LLM Reasoning through Logic Grid Puzzles (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to eliminate implicit biases in LLMs do not eradicate underlying behavioral bias. |
| Approach: | They propose a framework that uses logic grid puzzles to probe the influence of social stereotypes on logical reasoning and decision making in LLMs. |
| Outcome: | The proposed framework systematically probes the influence of social stereotypes on logical reasoning and decision making in LLMs. |
HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Autoregressive decoding limits the inference throughput of Large Language Models due to its sequential dependency. |
| Approach: | They propose a framework that allocates verification effort in proportion to candidate uncertainty. |
| Outcome: | Speculative decoding achieves an average speedup over state-of-the-art methods . a small subset of high-confidence predictions accounts for most successful verifications . |
ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current commonsense-reasoning tasks are discriminative in nature, where a model answers a multiple-choice question for a certain context. |
| Approach: | They propose a generative task that generates a commonsense-augmented graph for stance prediction by using a create-verify-and-refine graph collection framework. |
| Outcome: | The proposed model is able to generate a graph that serves as non-trivial, complete, and unambiguous explanation for the predicted stance. |
TeSum: Human-Generated Abstractive Summarization Corpus for Telugu (2022.lrec-1)
Copied to clipboard
| Challenge: | a number of recent datasets for summarisation, scraped the web-content relying on the assumption that summary is made available with the article by the publishers. |
| Approach: | They propose a pipeline that crowd-sources summarization data and then aggressively filters the content via: automatic and partial expert evaluation. |
| Outcome: | The proposed pipeline can be applied to scraped datasets to extract better quality articles-summaries pairs. |
Planning-Driven Programming: A Large Language Model Programming Workflow (2025.acl-long)
Copied to clipboard
| Challenge: | Recent research suggests continuous program refinements through visible tests to improve code generation accuracy in large language models (LLMs). |
| Approach: | They propose an LLM programming workflow to improve both initial code generation and subsequent refinements within a structured two-phase workflow. |
| Outcome: | The proposed workflow improves both initial code generation and subsequent refinements within a structured two-phase workflow. |
Hierarchical Evidence Set Modeling for Automated Fact Extraction and Verification (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact extraction and verification combine all evidence sentences to produce redundant information. |
| Approach: | They propose a framework to extract evidence sets and verify a claim to be supported, refuted or not enough info . they propose to encode and attend the claim and evidence sets at different levels of hierarchy . |
| Outcome: | The proposed framework outperforms 7 state-of-the-art methods for fact extraction and verification. |
Program Enhanced Fact Verification with Verbalization and Graph Attention Network (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fact verification based on structured data are challenging and require further study. |
| Approach: | They propose a program-enhanced verbalization and a graph attention network to integrate programs and execution into textual inference models. |
| Outcome: | The proposed framework achieves a new state-of-the-art accuracy on a benchmark dataset . it is compared with existing frameworks on symbolic and informal inference models . |
ContextCheck: Sentence-Level Faithfulness Verification with Context-Aware Disambiguation (2026.findings-acl)
Copied to clipboard
Yueqin Yin, Yaxi Li, Xin Liu, Xun Wang, Kaiqiang Song, Simin Ma, Shujian Liu, Sathish Reddy Indurthi, Haoyun Deng, Pengcheng He, Mingyuan Zhou, Song Wang
| Challenge: | Large language models often hallucinate, producing content that is factually incorrect or not grounded in the sources. |
| Approach: | They propose a framework for sentence-level faithfulness verification with context-aware disambiguation. |
| Outcome: | The proposed framework improves Macro F1 by over 10 points compared to baselines on three context-dependent datasets. |
FinChain: A Symbolic Benchmark for Verifiable Chain-of-Thought Financial Reasoning (2026.acl-long)
Copied to clipboard
Zhuohan Xie, Daniil Orel, Rushil Thareja, Dhruv Sahnan, Hachem Madmoun, Fan Zhang, Debopriyo Banerjee, Georgi Nenkov Georgiev, Xueqing Peng, Lingfei Qian, Jimin Huang, Jinyan Su, Aaryamonvikram Singh, Rui Xing, Rania Elbadry, Chen Xu, Haonan Li, Fajri Koto, Ivan Koychev, Tanmoy Chakraborty, Yuxia Wang, Salem Lahlou, Veselin Stoyanov, Sophia Ananiadou, Preslav Nakov
| Challenge: | Existing benchmarks emphasize final numerical answers while neglecting intermediate reasoning steps. |
| Approach: | They propose a symbolic benchmark for verifiable Chain-of-Thought evaluation in finance . FINCHAIN spans 58 topics across 12 financial domains and three difficulty levels . |
| Outcome: | The proposed benchmark aims to bridge symbolic reasoning and factual verification. |
PANDA: Persona Attributes Navigation for Detecting and Alleviating Overuse Problem in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | In persona-grounded dialogue, it is required to respond fluently and ground attributes according to the current conversation topic properly. |
| Approach: | They propose a framework to quantify the persona overuse problem of LLMs by establishing clear standards and verifying various LLM based on them. |
| Outcome: | The proposed framework aims to quantify the persona overuse problem of LLMs by establishing clear standards and verifying various LLM based on them. |
MM-Verify: Enhancing Multimodal Reasoning with Chain-of-Thought Verification (2025.acl-long)
Copied to clipboard
| Challenge: | MM-Verifier and MM Reasoner are a powerful multimodal reasoning model . large language models (LLMs) have demonstrated exceptional performance across tasks spanning myriad domains. |
| Approach: | They propose a method which combines tree search and verification to generate high-quality chain-of-thought data. |
| Outcome: | The proposed method outperforms all larger models on the MathCheck, MathVista, and MathVerse benchmarks. |
MedScore: Generalizable Factuality Evaluation of Open-ended Long-form Medical Answers by Domain-adapted Claim Decomposition and Verification (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing factuality evaluation pipelines are poor matches for medical domains . existing methods are limited to objective, entity-centric, formulaic texts . |
| Approach: | They propose a pipeline to decompose medical answers into condition-aware valid facts . they use a decomposition-then-verify approach to evaluate generated text . |
| Outcome: | The proposed method extracts up to three times as many valid facts as existing methods . the resulting factuality score substantially varies by decomposition method, corpus, and used backbone LLM . |
Agentic Rubrics as Contextual Verifiers for SWE Agents (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have rapidly advanced on coding tasks, enabling increasingly capable software engineering agents for real-time code editing and bug fixing. |
| Approach: | They propose to use a rubric checklist to create a context-grounded rubric for SWE agents. |
| Outcome: | The proposed rubrics achieve a score of 54.2% on Qwen3-Coder-30B-A3B and 40.6% on Qween3-332B . |
Act-Adaptive Margin: Dynamically Calibrating Reward Models for Subjective Ambiguity (2026.acl-long)
Copied to clipboard
Feiteng Fang, Dingwei Chen, Xiang Huang, Ting-En Lin, Yuchuan Wu, Xiong Liu, Jing Ye, Ziqiang Liu, Haonan Zhang, Liang Zhu, Hamid Alinejad-Rokny, Min Yang, Yongbin Li
| Challenge: | Existing approaches to reward modeling in reinforcement learning tasks are limited when dealing with ambiguous preferences. |
| Approach: | They propose to use AAM to dynamically calibrate preference margins using the Bradley-Terry model's internal parameter knowledge to improve reward modeling in subjective tasks. |
| Outcome: | The proposed approach improves reward modeling by dynamically calibrating preference margins using the model’s internal parameter knowledge. |
Process-Supervised Reinforcement Learning for Code Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing reinforcement learning strategies based on outcome supervision have shown effectiveness in code generation tasks, but their effectiveness in the field of code generation remains limited. |
| Approach: | They propose a method that uses a teacher model to mutate and refactor statements and a compiler to automatically label them. |
| Outcome: | The proposed method improves performance in complex code generation tasks. |
How Speculative Can Speculative Decoding Be? (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a largely increased latency due to their ability to autoregressively model . speculative decoding is a technique that trades generation quality for speed . |
| Approach: | They propose to use a draft model to draft tokens autoregressively and then verify them in parallel. |
| Outcome: | The proposed model could draft tokens autoregressively and then verify them in parallel . the proposed model trades quality for speed and could fail in verification stage . |
Revisiting Pathologies of Neural Models under Input Reduction (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that modern neural models tend to be miscalibrated. |
| Approach: | They examine why models produce high-confidence predictions on inputs that appear nonsensical to humans . previous work suggested that models fail to assign low probabilities due to model overconfidence . |
| Outcome: | The proposed methods can be extended to reduce the number of examples but with the cost of miscalibration. |
Denoising Rationalization for Multi-hop Fact Verification via Multi-granular Explainer (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing rationalization methods for multi-hop fact verification lack nuanced composition in the evidence, which leads to noise rationalization. |
| Approach: | They propose a method to obtain rationale by completely removing subset of input without compromising verification accuracy. |
| Outcome: | The proposed method outperforms 12 baselines on three multi-hop fact verification datasets. |
Robust Data Watermarking in Language Models by Injecting Fictitious Knowledge (2025.findings-acl)
Copied to clipboard
| Challenge: | Data watermarking in language models injects traceable signals, such as specific token sequences or stylistic patterns, into copyrighted text, allowing copyright holders to track and verify training data ownership. |
| Approach: | They propose a data watermarking approach that injects coherent and plausible yet fictitious knowledge into training data using generated passages describing a fictious entity and its associated attributes. |
| Outcome: | The proposed method is designed to be memorized by the LLM, and that increasing their density, length, and diversity of attributes strengthens their memorization. |
LLM Critics Help Catch Bugs in Mathematics: Towards a Better Mathematical Verifier with Natural Language Feedback (2025.findings-acl)
Copied to clipboard
Bofei Gao, Zefan Cai, Runxin Xu, Peiyi Wang, Ce Zheng, Runji Lin, Keming Lu, Dayiheng Liu, Chang Zhou, Wen Xiao, Tianyu Liu, Baobao Chang
| Challenge: | Existing mathematical verifiers are trained with binary classification labels, which are not informative enough for the model to accurately assess the solutions. |
| Approach: | They propose a natural language feedback-enhanced verifier that can validate the correctness of response generated by policy models by constructing automatically generated training data and a two-stage training paradigm. |
| Outcome: | The proposed verifier significantly improves in verification and reinforcement learning and alleviates data-demanding problems of the reward model. |
OpenFactCheck: Building, Benchmarking Customized Fact-Checking Systems and Evaluating the Factuality of Claims and LLMs (2025.coling-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Hasan Iqbal, Georgi N. Georgiev, Jiahui Geng, Iryna Gurevych, Preslav Nakov
| Challenge: | Large language models (LLMs) generate naturallysounding answers over a broad range of human inquiries, but they still produce content that deviates from real-world facts. |
| Approach: | They propose a framework for building customized automatic fact-checking systems, benchmarking their accuracy, evaluating factuality of LLMs, and verifying claims in a document. |
| Outcome: | The proposed framework assesses the factuality of free-form responses in open domains and evaluates factually of LLMs. |
Lightweight Query Checkpoint: Classifying Faulty User Queries to Mitigate Hallucinations in Large Language Model Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Question Answering (QA) with large language models has shown impressive performance, yet hallucinations persist . incorporating LQC into QA pipelines reduces hallucinosities while preserving strong answer quality. |
| Approach: | They propose a small classification model that detects verification-required queries before the LLM generates a potentially faulty answer. |
| Outcome: | The proposed model reduces hallucinations while maintaining strong answer quality. |
How to Train Your Fact Verifier: Knowledge Transfer with Multimodal Open Models (2024.findings-emnlp)
Copied to clipboard
Jaeyoung Lee, Ximing Lu, Jack Hessel, Faeze Brahman, Youngjae Yu, Yonatan Bisk, Yejin Choi, Saadia Gabriel
| Challenge: | a growing influx of misinformation across news and social media is hampered by outdated foundation model training data. |
| Approach: | They propose to use large language models to scale up online policing mechanisms . they evaluate foundation model performance without continual updating . |
| Outcome: | The proposed model can improve performance without continual updating . the proposed model improves on two widely used benchmarks . |
CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrieval (2026.acl-long)
Copied to clipboard
| Challenge: | Automated Fact-Checking has largely focused on verifying general knowledge against static corpora. |
| Approach: | They propose a benchmark to verify colloquial legal claims against Supreme Court precedents . the benchmark leverages large language models to synthesize claims from expert case summaries . they say the benchmark is a step forward in the field of legal fact verification . |
| Outcome: | The proposed benchmark bridges the gap between layperson assertions and technical jurisprudence while accounting for temporal validity. |
ClaimVer: Explainable Claim-Level Verification and Evidence Attribution of Text Through Knowledge Graphs (2024.findings-emnlp)
Copied to clipboard
Preetam Prabhu Srikar Dammu, Himanshu Naidu, Mouly Dewan, YoungMin Kim, Tanya Roosta, Aman Chadha, Chirag Shah
| Challenge: | Despite the fact that many fact-checking tools lack granularity and explainability, they lack the ability to be useful in various contexts. |
| Approach: | They propose a text validation framework that provides granular explanations for each claim and localizes the specific problematic content to reduce cognitive load. |
| Outcome: | The proposed framework provides granular explanations for each claim prediction and localizes and educates users on the specific content. |
EXPLAIN, EDIT, GENERATE: Rationale-Sensitive Counterfactual Data Augmentation for Multi-hop Fact Verification (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to augment training data with counterfactuals fail to handle multi-hop fact verification due to their incapability to preserve complex logical relationships. |
| Approach: | They propose to augment training data with counterfactuals that alter causal features of the original data by preserving logical relationships. |
| Outcome: | The proposed method outperforms the baselines and can generate linguistically diverse counterfactuals without disrupting their logical relationships. |
mCSQA: Multilingual Commonsense Reasoning Dataset with Unified Creation Strategy by Language Models and Humans (2024.findings-acl)
Copied to clipboard
| Challenge: | Currently, multilingual datasets are created through translation, which cannot evaluate such language-specific aspects. |
| Approach: | They propose to curate a dataset for language-specific knowledge and commonsense . they propose to use multilingual commonsensiaq to leverage language models for a more efficient construction . |
| Outcome: | The proposed method reduces the creation cost by using multilingual LMs to create QAs . the proposed approach is based on the construction process of CSQA but with language models . |
Residualized Similarity for Faithfully Explainable Authorship Verification (2025.findings-emnlp)
Copied to clipboard
Peter Zeng, Pegah Alipoormolabashi, Jihu Mun, Gourab Dey, Nikita Soni, Niranjan Balasubramanian, Owen Rambow, H. Schwartz
| Challenge: | Neural methods achieve high accuracy, but their representations lack direct interpretability. |
| Approach: | They propose a method that supplements systems using interpretable features with a neural network to improve their performance while maintaining interpretability. |
| Outcome: | The proposed method improves the performance of state-of-the-art models while maintaining interpretability. |
Faithful and Robust LLM-Driven Theorem Proving for NLI Explanations (2025.acl-long)
Copied to clipboard
| Challenge: | Recent work has shown that the interaction of large language models (LLMs) with theorem provers (TPs) can help verify and improve the validity of NLI explanations. |
| Approach: | They propose to use logical expressions to guide LLMs in generating structured proof sketches and to use them to improve their accuracy. |
| Outcome: | The proposed strategies improve autoformalisation, syntactic errors and explanation refinement over the state-of-the-art model. |
Double: Breaking the Acceleration Limit via Double Retrieval Speculative Parallelism (2026.acl-long)
Copied to clipboard
| Challenge: | Parallel Speculative Decoding (PSD) has limitations due to speedup limits and high computational waste . a novel synchronous mechanism solves the Retrieval Precision-Efficiency Dilemma . |
| Approach: | They propose a framework that combines a draft-verification-based approach with a synchronous mechanism to solve the Retrieval Precision-Efficiency Dilemma. |
| Outcome: | The proposed framework breaks speedup limits for Speculative Decoding by overlapping draft generation with verification. |
AttributionBench: How Hard is Automatic Attribution Evaluation? (2024.findings-acl)
Copied to clipboard
| Challenge: | generative search engines enhance the reliability of large language model responses by providing cited evidence. |
| Approach: | They propose to use a benchmark to evaluate whether a large language model supports the generated responses or not . |
| Outcome: | The proposed benchmark shows that even a fine-tuned GPT-3.5 only achieves around 80% macro-F1 under a binary classification formulation. |
VerifiAgent: a Unified Verification Agent in Language Model Reasoning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing verification methods are typically model-specific or domain-restricted . Existing methods require significant computational resources and lack scalability . |
| Approach: | a unified verification agent integrates two levels of verification: meta-verification and tool-based adaptive verification. |
| Outcome: | The proposed agent outperforms baseline verification methods among reasoning tasks. |
VeriFact: Enhancing Long-Form Factuality Evaluation with Refined Fact Extraction and Reference Facts (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work focuses on accuracy and precision, but factuality evaluation is difficult due to inter-sentence dependencies. |
| Approach: | They introduce a factuality evaluation framework to enhance fact extraction . they also introduce 'factRBench' that evaluates both precision and recall . |
| Outcome: | The proposed framework enhances fact extraction by identifying incomplete and missing facts . it also evaluates precision and recall in long-form models, whereas prior work focuses on precision. |
LI4: Label-Infused Iterative Information Interacting Based Fact Verification in Question-answering Dialogue (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing studies on fact verification have failed to fully exploit question structures and ignoring relevant label information during the verification process. |
| Approach: | They propose a new approach for question-answering dialogue based fact verification using label-infused iterative information interacting. |
| Outcome: | The proposed approach achieves remarkable performance on HEALTHVER, FAVIQ, and COLLOQUIAL. |
LM2: A Simple Society of Language Models Solves Complex Reasoning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that providing guidance via decomposing the original question into multiple subproblems elicits more robustness in LLM reasoning. |
| Approach: | They propose a language-based decomposition, solution and verification framework that modularizes the decomposer, solution, and verification into three different language models. |
| Outcome: | The proposed model outperforms existing methods on in- and out-domain reasoning problems, outperforming the best baselines by 8.1% on MATH, 7.71% on JEEBench, and 9.7% on MedQA problems. |
FactLens: Benchmarking Fine-Grained Fact Verification (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive capability in language generation and understanding, but their tendency to hallucinate and produce factually incorrect information remains a key limitation. |
| Approach: | They propose a benchmark to evaluate fine-grained fact verification where claims are broken down into smaller sub-claims for individual verification. |
| Outcome: | The proposed model enables more precise identification of inaccuracies, improved transparency, and reduced ambiguity in evidence retrieval. |
Beyond Ranking: Fine-Grained Diagnostics and Self-Improvement for MLLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Current paradigms rely on holistic scoring and static leaderboards to disentangle fine-grained competencies. |
| Approach: | They propose a framework to shift the focus from ranking to fine-grained diagnosis. |
| Outcome: | The proposed framework surpasses the strongest baseline by 7.92%. |
GUITester: Enabling GUI Agents for Exploratory Defect Discovery (2026.findings-acl)
Copied to clipboard
| Challenge: | Exploratory GUI testing is essential for software quality but suffers from high manual costs. |
| Approach: | They propose a framework that decouples navigation from verification via two modules . they propose 143 tasks and a GUITestBench benchmark that features 26 defects . |
| Outcome: | The proposed framework outperforms state-of-the-art benchmarks in 143 tasks and 26 defects. |
Following Occam’s Razor: Dynamic Combination of Structured Knowledge for Multi-Hop Question Answering using LLMs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Multi-hop question answering is a challenging task that requires capturing information from multiple positions in multiple documents. |
| Approach: | They propose a framework for integrating text-based and triple-based paradigms that incorporates structured knowledge into large-scale question answering. |
| Outcome: | The proposed framework improves multi-hop question answering by incorporating structured knowledge into the models. |
MolErr2Fix: Benchmarking LLM Trustworthiness in Chemistry via Modular Error Detection, Localization, Explanation, and Correction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown growing potential in molecular sciences, but they often produce chemically inaccurate descriptions and struggle to recognize or justify potential errors. |
| Approach: | They propose a benchmark to assess LLMs on error detection and correction in molecular descriptions. |
| Outcome: | The proposed benchmark targets LLMs on error detection and correction in molecular descriptions. |
Rethinking LLM Watermark Detection in Black-Box Settings: A Non-Intrusive Third-Party Framework (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing secret-key schemes tightly couple detection with injection . this dependency creates a fundamental barrier for real-world governance . |
| Approach: | et al. introduce a black-box framework for non-intrusive, third-party watermark verification . they propose a proxy model to amplify watermark-relevant signals and complementary relative measurements . |
| Outcome: | a new framework decouples detection from injection and assesses alignment of query text with watermark distributions. |
Zero-Shot Fact Verification via Natural Logic and Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advancements in fact verification systems with natural logic have enhanced their explainability by aligning claims with evidence through set-theoretic operators, providing faithful justifications. |
| Approach: | They propose a method that utilizes the generalization capabilities of instruction-tuned large language models to provide faithful justifications. |
| Outcome: | The proposed method outperforms other systems that were not specifically trained on natural logic data, and achieves an average accuracy improvement of 8.96 points over the baseline. |
Veri-R1: Toward Precise and Faithful Claim Verification via Online Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to online claim verification rely on prompt engineering or pre-designed reasoning workflows. |
| Approach: | They propose an online reinforcement learning framework that enables an LLM to interact with a search engine and receive reward signals that explicitly shape its planning, retrieval, and reasoning behaviors. |
| Outcome: | Empirical results show that Veri-R1 improves joint accuracy by 30% and doubles evidence score, often surpassing larger-scale model counterparts. |
ML-Promise: A Multilingual Dataset for Corporate Promise Verification (2025.emnlp-main)
Copied to clipboard
| Challenge: | Promises shape perceptions and drive decisions, but verification of their fulfillment is difficult due to complexity and volume of commitments . authors propose a new approach to verifying promises in environmental, social, and governance reports . complexity of promises, complexity of evidence, difficulty in verifying their fulfillment a pressing need for new approaches . |
| Approach: | They propose a multilingual dataset that includes English, French, Chinese, Japanese, and Korean . they propose ML-Promise to facilitate in-depth verification of corporate promises . |
| Outcome: | The proposed approach includes promise identification, evidence assessment, and evaluation of timing for verification in multiple languages. |
SPIDE: Serial and Parallel Intertwined Speculative Decoding (2026.findings-acl)
Copied to clipboard
Wenru Xu, Peixuan Xu, Ziqi Yang, Ming Hu, Zihui Wang, Jianzhong Qi, Rongshan Yu, Xiaoliang Fan, Cheng Wang
| Challenge: | Speculative decoding (SD) is a training-free SD framework that orchestrates dynamic alternation combining serial dynamic drafting with parallel draft verification. |
| Approach: | They propose a serial and parallel intertwined speculative DEcoding framework that orchestrates dynamic alternation combining serial dynamic drafting and parallel draft verification. |
| Outcome: | The proposed framework accelerates inference while reducing the LLM usage costs. |
PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving (2025.emnlp-main)
Copied to clipboard
Mihir Parmar, Xin Liu, Palash Goyal, Yanfei Chen, Long Le, Swaroop Mishra, Hossein Mobahi, Jindong Gu, Zifeng Wang, Hootan Nakhost, Chitta Baral, Chen-Yu Lee, Tomas Pfister, Hamid Palangi
| Challenge: | Existing methods for natural planning lack constraint-guided iterative verification and adaptive selection . a recent study found that LLMs are not good at such planning. |
| Approach: | They propose a model-agnostic and easily scalable agent framework with three key components: constraint, verification, and selection agents. |
| Outcome: | The proposed framework improves inference-time algorithms on NATURAL PLAN and OlympiadBench benchmarks. |
Enhancing Mathematical Reasoning in LLMs by Stepwise Correction (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Best-of-N decoding methods often lead to incorrect solutions . a novel method is proposed to help large language models identify and revise incorrect steps in their generated reasoning paths. |
| Approach: | They propose a method that helps large language models identify and revise incorrect steps in their generated reasoning paths. |
| Outcome: | The proposed method outperforms the state-of-the-art Best-ofN decoding method by +2.4 and reduces token consumption by 77.8%. |
Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification (2025.findings-acl)
Copied to clipboard
John Dougrez-Lewis, Mahmud Elahi Akhter, Federico Ruggeri, Sebastian Löbbers, Yulan He, Maria Liakata
| Challenge: | Large Language Models (LLMs) have shown remarkable proficiency in complex tasks where reasoning capabilities are paramount. |
| Approach: | They propose a framework to break down claims into atomic reasoning types needed for verification. |
| Outcome: | The proposed framework breaks down claims into atomic reasoning types needed for verification. |
Semantic-pragmatic Annotations in the Prague Dependency Treebank (2026.findings-acl)
Copied to clipboard
Marie Mikulová, Eva Hajicova, Jiří Mírovský, Anna Nedoluzhko, Michal Novák, Pavlína Synková, Jan Štěpánek, Barbora Štěpánková, Jan Hajič
| Challenge: | morphology and syntax work on sentence level, but semantic-pragmatic phenomena are often related to two or more neighbouring sentences and possibly to an extra-linguistic context. |
| Approach: | They present semantic-pragmatic specification and annotations in the Prague Dependency Treebank - Consolidated 2.0 release2 by annotating the entire corpus. |
| Outcome: | The proposed annotations are based on the Prague Dependency Treebank -Consolidated 2.0 (PDT-C 2.0) the dataset contains more than 3 million tokens (of Czech) manually annotated from morphology to surface and deep syntax including several types of semantic-pragmatic annotations. |
Training-Free Adaptive Speculative Decoding via Linguistic Priors (2026.findings-acl)
Copied to clipboard
| Challenge: | Speculative decoding (SPD) is a promising technique to accelerate Large Language Models (LLMs). current approaches neglect the inherent heterogeneity of natural language and fail to distinguish between semantically-rich content and structurally-predictable syntax. |
| Approach: | They propose a training-free framework that leverages linguistic priors to enable adaptive drafting and verification. |
| Outcome: | The proposed framework significantly accelerates inference without additional training. |
DataNarrative: Automated Data-Driven Storytelling with Visualizations and Texts (2024.emnlp-main)
Copied to clipboard
| Challenge: | Data-driven storytelling uses visual aids and visualizations to convey insights. |
| Approach: | They propose a task for data story generation using large language models and a benchmark containing 1,449 stories from diverse sources. |
| Outcome: | The proposed framework outperforms non-agentic counterparts in both model-based and human evaluations, but also reveals unique challenges in data story generation. |
ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interaction (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are limited by knowledge cutoff and can generate factual hallucinations when handling time-sensitive news. |
| Approach: | They propose a two-stage zero-shot fake news detection framework that uses a hierarchical salience and saliency-calibrated minimum margin of relevance algorithm to extract core entities accurately. |
| Outcome: | The proposed framework outperforms existing zero-shot baselines and even most few-shot methods on two public datasets. |
Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing CoT compression methods struggle to balance accuracy and efficiency . long CoT reasoning also introduces an overthinking phenomenon, authors say . |
| Approach: | They propose a framework that performs step-wise CoT compression by modeling stage-specific redundancy sources and integrating with a retrieval-augmented guidance. |
| Outcome: | The proposed framework reduces average response length by 59.9% while improving accuracy by 4.8 points over existing methods. |
Marco-Bench-MIF: On Multilingual Instruction-Following Capability of Large Language (2025.acl-long)
Copied to clipboard
Bo Zeng, Chenyang Lyu, Sinuo Liu, Mingyan Zeng, Minghao Wu, Xuanfan Ni, Tianqi Shi, Yu Zhao, Yefeng Liu, Chenyu Zhu, Ruizhe Li, Jiahui Geng, Qing Li, Yu Tong, Longyue Wang, Weihua Luo, Kaifu Zhang
| Challenge: | Existing datasets for instruction-following are monolingual and centered on English . existing data are unable to capture linguistic and cultural subtle differences . |
| Approach: | They propose an extension of IFEval to a localized multilingual version called Marco-Bench-MIF . their benchmark addresses linguistic constraints and cultural references via translation and verification . |
| Outcome: | The proposed extension of IFEval to a localized multilingual version covers 30 languages with varying levels of localization. |
ImF: Embedding an Implicit Fingerprint in Your Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Training and serving large language models (LLMs) is resource-intensive, making reliable intellectual property protection and black-box ownership verification increasingly important. |
| Approach: | They propose a method to inject a small set of secret query–response behaviors into model fingerprinting . they encode ownership information into a natural-looking target response and derive a semantically aligned query . |
| Outcome: | The proposed fingerprints improve stealthiness and remain verifiable under model updates and deployment-time prompt interventions. |
Temporal Consistency for LLM Reasoning Process Error Identification (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical evaluations show consistent performance improvements over baseline methods . 7B/8B distilled models outperform all 70B/72B models and GPT-4o on ProcessBench . |
| Approach: | They propose a temporal consistency method that leverages consistency in a sequence of self-reflection actions to improve verification accuracy. |
| Outcome: | The proposed method outperforms existing methods on three benchmarks . it leverages consistency in a sequence of self-reflection actions to improve accuracy . |
DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent measures of factual precision use a decompose-then-verify framework . decontextualization is the process of augmenting subclaims with necessary context . |
| Approach: | They evaluate different decomposition, decontextualization and verification strategies . they introduce a deconstructualization aware verification method that validates subclaims in context . |
| Outcome: | The proposed method decomposes claims and independently verifyes them . it introduces a decontextualization aware verification method that validates subclaims in context . |
ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to chain-of-thought reasoning incur high inference latency due to long generation traces. |
| Approach: | They propose a confidence-gated cascaded verification framework that reduces the trade-off between generation and verification. |
| Outcome: | The proposed framework achieves 2.24 speedups while matching target-model accuracy. |
Deconstruct, Diagnose, and Deliberate: A Protocol-Adaptive Role-Specific Multi-Agent Framework for Fake News Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for fake news detection rely on monolithic verification methods . Existing approaches often yield ambiguous verdicts due to superficial processing . |
| Approach: | They propose a protocol-adaptive role-specific multi-agent framework that decomposes verification into factual, logical, and contextual dimensions. |
| Outcome: | The proposed framework outperforms baseline methods in both predictive accuracy and explanatory quality. |
Identifying and Answering Questions with False Assumptions: An Interpretable Approach (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate misleading answers because of hallucinations . despite their capabilities, LLMs suffer from hallucinisms, which leads to unfaithful answers . |
| Approach: | They propose a method to identify and answer questions with false assumptions . they first investigate whether the problem reduces to fact verification . then, they leverage external evidence to mitigate hallucinations . |
| Outcome: | The proposed approach reduces the problem to fact verification and provides interpretable answers by pinpointing the false assumptions. |
Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification (2026.findings-acl)
Copied to clipboard
Yuxuan Wan, Tianqing Fang, Zaitang LI, Yintong Huo, Wenxuan Wang, Haitao Mi, Dong Yu, Michael R. Lyu
| Challenge: | Recent advances in Deep Research Agents (DRAs) are transforming automated knowledge discovery and problem-solving. |
| Approach: | They propose an inference-time scaling of verification wherein an agent self-improves at test time by evaluating its generated answers. |
| Outcome: | The proposed model outperforms vanilla agent-as-judge and LLM judge baselines by 12%–48% in meta-evaluation F1 score. |
Neo-Classic: A Benchmark for Evaluating Linguistic-Aesthetic Reasoning in Classical Chinese Poetry (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) achieve high accuracy on established Classical Chinese Poetry benchmarks, but it remains challenging to distinguish transferable Linguistic-Aesthetic Reasoning from reliance on familiar pre-training patterns. |
| Approach: | They propose a benchmark that combines a constructionist Out-of-Sample dataset with reverse understanding probes to evaluate large-scale large-format models. |
| Outcome: | The proposed model performs well on classical Chinese poetry benchmarks, but a performance gap persists . the model can complete famous couplets and can be used to understand a variety of texts. |
Multilingual and Cross-Lingual Citation Needed Detection on Wikipedia for Lower-Resource Languages (2026.acl-long)
Copied to clipboard
| Challenge: | Existing research has largely overlooked lower-resource languages for automated fact-checking. |
| Approach: | They propose a multilingual CND corpus spanning 18 languages across three resource levels and a small decoder-based language model for CND. |
| Outcome: | The proposed model outperforms prompted LLMs in cross-lingual CND across languages. |
FaStFact: Faster, Stronger Long-Form Factuality Evaluations in LLMs (2025.findings-emnlp)
Copied to clipboard
Yingjia Wan, Haochen Tan, Xiao Zhu, Xinyu Zhou, Zhiwei Li, Qingsong Lv, Changxuan Sun, Jiaqi Zeng, Yi Xu, Jianqiao Lu, Yinhong Liu, Zhijiang Guo
| Challenge: | Prior evaluation pipelines fail to evaluate factuality of long-form LLMs due to inefficiency and costly human assessment. |
| Approach: | They propose a fast and strong evaluation pipeline that can evaluate factuality of long-form LLMs . they propose 'faStFact' to reduce cost of web searching and inference calling . |
| Outcome: | The proposed evaluation pipeline achieves highest alignment with human evaluation and efficiency among existing baselines. |
ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning (2025.emnlp-main)
Copied to clipboard
Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Deli Zhao, Wenbing Huang, Tingyang Xu, Qifeng Bai, Yu Rong
| Challenge: | Existing medical reasoning datasets are limited in scale and typically rely on incomplete data. |
| Approach: | They propose to use ReasonMed to train medical reasoning models using a multi-agent generation, verification, and refinement pipeline. |
| Outcome: | The largest medical reasoning dataset to date surpasses the prior best sub-10B models by 4.17% and even exceeds LLaMA3.1-70B on PubMedQA by 4.60%. |
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference (2026.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) is a powerful and efficient way to accelerate autoregressive generation. |
| Approach: | They propose a training-free framework that recovers valid tokens discarded by standard verification . they use online correction memory and Semantic Consistency Gating to analyze rejections . |
| Outcome: | The proposed framework outperforms existing methods and achieves peak throughput speedup of 2.33x. |
MavenCoder: Competitive Code Generation via Model Adaptive Planning Strategies and Multi-Perspective Verification Enhancement (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have significantly enhanced automated program synthesis. |
| Approach: | They propose a model-adaptive and verification–enhanced framework for competition-level code generation that leverages adaptive assessment aligned with the model’s capabilities to select planning strategies while providing timely feedback and correction via multi-perspective verification. |
| Outcome: | The proposed framework outperforms existing state-of-the-art approaches on livecodebench, humanEval+, MBPP+, and codecontests, and achieves pass@1 results exceeding 3%–40%. |
Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to reasoning faithfulness violate constraints, authors say . a science fantasy series and companion books are among the books . |
| Approach: | They propose a framework that enforces verification over internal belief states within the agent before action commitment, achieving faithful reasoning. |
| Outcome: | The proposed framework improves reasoning faithfulness while preserving competitive end-task performance. |
RubricHub: A Comprehensive and Highly Discriminative Rubric Dataset via Automated Coarse-to-Fine Generation (2026.acl-long)
Copied to clipboard
Sunzhu Li, Jiale Zhao, Huimin Ren, Zhenlin Wei, Yang Zhou, Jingwen Yang, Shunyu Liu, Kaike Zhang, Chen Wei
| Challenge: | Existing methods for generating open-ended rubrics suffer from scalability bottlenecks and coarse criteria resulting in a supervision ceiling effect. |
| Approach: | They propose a framework for automated Coarse-to-Fine Rubric Generation . their framework uses principle-guided synthesis, multi-model aggregation, difficulty evolution . |
| Outcome: | The proposed framework produces comprehensive and highly discriminative criteria capable of capturing the subtle nuances. |
Large Language Models Require Curated Context for Reliable Political Fact-Checking—Even with Reasoning and Web Search (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have raised hopes for automated end-to-end fact-checking, but prior studies report mixed results. |
| Approach: | They evaluate 15 large language models on 6,000 claims fact-checked by PolitiFact . standard models perform poorly, reasoning offers minimal benefits, and web search provides only moderate gains . |
| Outcome: | The models predict claim veracity and a curated RAG system improved macro F1 by 233% on average across model variants. |
TSVer: A Benchmark for Fact Verification Against Time-Series Evidence (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing systems for fact-checking lack structured evidence, provide insufficient justifications for verdicts, or rely on synthetic claims. |
| Approach: | They propose a temporal and numerical reasoning dataset based on time-series evidence that is annotated with time frames and a verdict and justifications reflecting how the evidence is used to reach the verdict. |
| Outcome: | The proposed dataset improves the quality of the annotations and achieves an inter-annotator agreement of = 0.745 on verdicts. |
CLaSp: In-Context Layer Skip for Self-Speculative Decoding (2025.acl-long)
Copied to clipboard
Longze Chen, Renke Shan, Huiming Wang, Lu Wang, Ziqiang Liu, Run Luo, Jiawei Wang, Hamid Alinejad-Rokny, Min Yang
| Challenge: | Existing methods for drafting Large Language Models require additional modules to be trained, which can be challenging to implement and ensure compatibility across various LLMs. |
| Approach: | They propose an in-context layer-skipping strategy for self-speculative decoding that uses a plug-and-play mechanism to skip intermediate layers of the verify model to construct a compressed draft model. |
| Outcome: | The proposed method achieves a speedup of 1.3 1.7 on LLaMA3 series models without altering the original distribution of the generated text. |
Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Model (2025.acl-long)
Copied to clipboard
| Challenge: | Existing Chain-of-Thought (CoT) methods struggle with consistency and verification in complex reasoning tasks. |
| Approach: | They propose a framework that integrates structured knowledge representation with learned planning. |
| Outcome: | The proposed framework outperforms existing Chain-of-Thought (CoT) methods on math reasoning, logical reasoning, and coding tasks. |
ClaimDB: A Fact Verification Benchmark over Large Structured Data (2026.acl-long)
Copied to clipboard
| Challenge: | despite substantial progress in fact-verification benchmarks, this setting remains largely underexplored. |
| Approach: | They propose a fact-verification benchmark where evidence for claims is derived from compositions of millions of records and multiple tables. |
| Outcome: | The proposed benchmarks score below 55% accuracy with 30 state-of-the-art LLMs and are released on github. |
The Missing Parts: Augmenting Fact Verification with Half Truth Detection (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing fact verification systems assess whether a claim is true or false . but many real-world claims are half-truths due to omission of critical context . a new framework that detects omitted information can improve existing fact-checking pipelines . |
| Approach: | They propose a framework that detects omission-based misinformation by aligning evidence and inferring implied intent. |
| Outcome: | The proposed framework boosts Half-True classification F1 by up to 16 points . it can be integrated into existing fact-checking pipelines and improves performance across strong baselines. |
Efficient Real-time Refinement of Language Model Text Generation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) generate factually incorrect answers, a challenge that remains . Streaming-VR enables on-the-fly verification and correction of tokens as they are generated . |
| Approach: | They propose a method that enables on-the-fly verification and correction of LLM tokens as they are generated. |
| Outcome: | The proposed method improves factual accuracy and improves refinement efficiency compared to prior methods. |
POSTCONDBENCH: Benchmarking Correctness and Completeness in Formal Postcondition Inference (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks emphasize correctness under limited evaluation settings . evaluation of formal specifications is time-consuming, errorprone and requires substantial expertise. |
| Approach: | They propose a multilingual benchmark for evaluating method-level postcondition generation from real-world software. |
| Outcome: | The proposed benchmarks show that evaluation remains a key bottleneck . 420 Python and Java tasks are paired with a high-quality postcondition set . |
Speculative Verification: Exploiting Information Gain for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used for many applications but their size and computational cost make inference serving a significant challenge. |
| Approach: | They propose an efficient augmentation to Speculative Decoding (SD) that predicts speculation accuracy and dynamically adapts the verification length to maximize throughput. |
| Outcome: | The proposed model reduces wasted verification on rejected tokens and improves decoding efficiency. |
Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models (2026.findings-acl)
Copied to clipboard
Can Xu, Lingyong Yan, Jiayi Wu, Haosen Wang, Shuaiqiang Wang, Yuchen Li, Jizhou Huang, Dawei Yin, Xiang Li
| Challenge: | Existing training paradigms rely on outcome-oriented rewards, which provide insufficient signal for shaping the complex, multi-step reasoning process. |
| Approach: | They propose a framework that integrates large reasoning models with retrieval-augmented generation to improve reasoning fidelity and verification rigor. |
| Outcome: | Experiments on multiple benchmarks demonstrate the effectiveness of the proposed framework. |
MirrorQA: Benchmarking Multimodal LLMs on Mirror-Orientation Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Multimodal large language models (MLLMs) have achieved remarkable progress in recent years, yet their ability to perform left–right reasoning in mirror contexts remains underexplored. |
| Approach: | They propose a benchmark to evaluate MLLMs' ability to distinguish left from right from a subject-centered perspective. |
| Outcome: | The proposed benchmarks show that even the best performing models achieve only 65.40% accuracy, far below the 99.28% accuracy of humans. |
Seeing Isn’t Believing: Mitigating Belief Inertia via Active Intervention in Embodied Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) have enabled agents to tackle complex embodied tasks through environmental interaction, but they still make suboptimal decisions and perform ineffective actions. |
| Approach: | They propose an active belief intervention mechanism that generates explicit belief states . they characterize belief inertia as a key failure mode of LLM-based agents . |
| Outcome: | The proposed method achieves significant gains in task success rates across embodied benchmarks. |
Knowledge-to-Verification: Exploring RLVR for LLMs in Knowledge-Intensive Domains (2026.acl-long)
Copied to clipboard
Zhonghang Yuan, Zhefan Wang, Fang Hu, Zihong Chen, Jinzhe Li, Gang Li, Jie Ying, Huanjun Kong, Songyang Zhang, Nanqing Dong
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable progress in reasoning, but their applications on knowledge-intensive domains have not been explored due to the scarcity of high-quality verifiable data. |
| Approach: | They propose a framework that extends reinforcement learning with verifiable rewards (RLVR) to knowledge-intensive domains through automated verififiability data synthesis while enabling verification of the LLM's reasoning process. |
| Outcome: | Extensive experiments show that the proposed framework enhances the reasoning of large language models in knowledge-intensive domains without significantly compromising the model’s general capabilities. |
Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks (2026.acl-long)
Copied to clipboard
Wenqi Zhang, Mengna Wang, Gangao Liu, Huixin Xu, Yiwei Jiang, Yongliang Shen, Guiyang Hou, Zhe Zheng, Hang Zhang, Xin Li, Jiajun Liu, Weiming Lu, Peng Li, Yueting Zhuang
| Challenge: | Recent advances in reasoning models have demonstrated remarkable capabilities on mathematical and coding tasks, but their effectiveness in embodied domains remains largely unexplored. |
| Approach: | They propose a reasoning model for interactive embodied tasks that synthesizes 9.3k coherent Observation-Thought-Action trajectories containing 64k ego-centric images and 90k diverse reasoning processes. |
| Outcome: | The proposed model outperforms existing visual reasoning models by +9%, 24%, and +13% on long-horizon tasks. |
Thinking Traps in Long Chain-of-Thought: A Measurable Study and Trap-Aware Adaptive Restart (2026.findings-acl)
Copied to clipboard
null Chenkang, Fan Yu, Junjie Nian, Sihan Zhao, Zhuoka Feng, Zijun Yao, Wang Heng, Yu Minshen, Yixin Cao
| Challenge: | Experiments show that extended generation does not guarantee correctness . a recurring pattern in Long-CoT failures is a problem for large reasoning models . |
| Approach: | They propose a test-time control framework that truncates the trajectory before the trap segment and adaptively restarts decoding. |
| Outcome: | Experiments show that TAAR improves reasoning performance without fine-tuning model parameters. |
Agentic Verification for Ambiguous Query Disambiguation (2026.findings-acl)
Copied to clipboard
Youngwon Lee, Seung-won Hwang, Ruofan Wu, Feng Yan, Danmei Xu, Moutasem Akkad, Zhewei Yao, Yuxiong He
| Challenge: | Prior Diversify-then-Verify pipelines generate interpretations and then retrieve evidence . ambiguous queries require RAG to disambiguate into interpretations that can be answered from corpus . |
| Approach: | They propose a novel approach that unifies diversification with verification by integrating retriever relevance and generator answerability feedback early. |
| Outcome: | The proposed approach improves grounding-aware F1 by 23% over baselines across multiple LLMs. |
SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering? (2026.findings-acl)
Copied to clipboard
Yuxuan Sun, Yuze Zhao, Yufeng Wang, Yao Du, Zhiyuan Ma, Jinbo Wang, Mengdi Zhang, Kai Zhang, Zhenya Huang
| Challenge: | Evaluating software engineering capabilities is a core component of large language models (LLMs). |
| Approach: | They propose a benchmark to evaluate LLM-generated test suites that introduces mutated solutions that attempt to "fool" them. |
| Outcome: | The proposed test suites are based on 2,636 mutated variants derived from 800 original instances and include a multilingual subset spanning nine programming languages. |
Beyond ’Aha!’: Toward Systematic Meta-Abilities Alignment in Large Reasoning Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has shown that outcome-based reinforcement learning (RL) can incidentally elicit advanced reasoning behaviors such as self-correction, backtracking, and verification. |
| Approach: | They explicitly align large reasoning models with three meta-abilities: deduction, induction, and abduction, using automatically generated, self-verifiable tasks. |
| Outcome: | The proposed model aligns models with deduction, induction, and abduction meta-abilities using automatically generated, self-verifiable tasks. |
Characterizing and Evaluating Working Emotion Vocabularies in Multilingual Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Prior work evaluating emotion and affective understanding in large language models rely on predetermined label sets or focus on a singular evaluation task. |
| Approach: | They examine the ability of multilingual language models to predict any term used by an author to label their own feelings or emotions. |
| Outcome: | The proposed models perform poorly on three different tasks in English and Spanish. |