Localizing Factual Inconsistencies in Attributable Text Generation (2026.tacl-1)
Copied to clipboard
Arie Cattan, Paul Roit, Shiyue Zhang, David Wan, Roee Aharoni, Idan Szpektor, Mohit Bansal, Ido Dagan
| Challenge: | Existing methods for detecting hallucinations in model-generated texts fail to pinpoint errors. |
| Approach: | They propose a formalism for localizing factual inconsistencies in attributable text generation . they propose to decompose the generated text into simple question-answer pairs . |
| Outcome: | The proposed method achieves substantial inter-annotator agreement while achieving a substantial consistency score. |
Similar Papers
Shortcomings of Question Answering Based Factuality Frameworks for Error Localization (2023.eacl-main)
Copied to clipboard
| Challenge: | Abstractive summarization systems often generate summaries with factual errors . many approaches to detect these errors have been proposed, but this capability has not been evaluated in past research . |
| Approach: | They propose to use question answering-based factuality metrics to detect errors in summaries . they find that QA-based frameworks fail to correctly identify error spans in generated summary . |
| Outcome: | The proposed methods outperform trivial exact match baselines in localizing errors in summaries. |
Open-World Factually Consistent Question Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for question generation suffer from factual inconsistencies and incorrect entities and are not answerable from the input paragraph. |
| Approach: | They propose a data processing technique based on de-lexicalization for consistent question generation across domains and a model that is generic across question-generation models. |
| Outcome: | The proposed method produces entity-level factually consistent questions without significant impact on traditional metrics. |
AlignScore: Evaluating Factual Consistency with A Unified Alignment Function (2023.acl-long)
Copied to clipboard
| Challenge: | Existing methods to evaluate factual consistency of text depend on limited data . e.g., generated text can contain factual inconsistencies that are irrelevant to context . |
| Approach: | They propose a new holistic metric that measures factual inconsistencies . they use 4.7M training examples from 7 well-established tasks . |
| Outcome: | The proposed metric outperforms existing metrics on 22 datasets and matches or outperFORMs them. |
TRUE: Re-evaluating Factual Consistency Evaluation (2022.naacl-main)
Copied to clipboard
Or Honovich, Roee Aharoni, Jonathan Herzig, Hagai Taitelbaum, Doron Kukliansy, Vered Cohen, Thomas Scialom, Idan Szpektor, Avinatan Hassidim, Yossi Matias
| Challenge: | Grounded text generation systems often generate factual inconsistencies, hindering their real-world applicability. |
| Approach: | They propose a method to assess factual consistency metrics on standardized texts . they recommend NLI and question generation-and-answering-based methods as starting points . |
| Outcome: | The proposed method is more actionable and interpretable than previous methods. |
Falsesum: Generating Document-level NLI Examples for Recognizing Factual Inconsistency in Summarization (2022.naacl-main)
Copied to clipboard
| Challenge: | Neural abstractive summarization models generate factually inconsistent summaries . previous work has introduced the task of recognizing factual inconsistency as a downstream application of natural language inference (NLI). |
| Approach: | They propose a data generation pipeline that enables a task-oriented approach to detect factual inconsistencies in abstractive summarization models. |
| Outcome: | The proposed model improves the state-of-the-art performance across four benchmarks for recognizing factual inconsistency in generated summaries. |
PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise (2026.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are prone to factually inconsistent statements, known as hallucinations. |
| Approach: | They propose to train a specialized model that detects inconsistencies over text prefixes to improve generation faithfulness by 5-14 F1 points. |
| Outcome: | The proposed model outperforms baseline models by 5-14 F1 points in prefix-level entailment. |
Semantic Consistency-Based Uncertainty Quantification for Factuality in Radiology Report Generation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Radiology report generation has shown great potential in assisting radiologists . generative medical Vision Large Language Models (VLLMs) are prone to hallucinations and can produce inaccurate diagnostic information. |
| Approach: | They propose a framework that provides both report-level and sentence-level uncertainties. |
| Outcome: | The proposed method improves factuality scores by 10% by rejecting 20% of reports on the MIMIC-CXR dataset. |
Evaluating Factuality in Generation with Dependency-level Entailment (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing text generation models produce text that is factually inconsistent with information in the input. |
| Approach: | They propose a new formulation of entailment that decomposes it at the level of dependency arcs. |
| Outcome: | The proposed model can identify factual errors better than sentence-level methods and question generation methods while localizing the erroneous parts of the generation. |
Annotating and Modeling Fine-grained Factuality in Summarization (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent abstractive summarization systems produce factual errors that are not faithful to the input . current methods are lacking in identifying what errors are most important to target . |
| Approach: | They use synthetic and human-labeled data to identify factual errors in summarization and train models on the factuality detection task. |
| Outcome: | The proposed model detects factual errors on word, dependency, and sentence levels. |
Ask, Assess, and Refine: Rectifying Factual Consistency and Hallucination in LLMs with Metric-Guided Feedback Learning (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have heralded unprecedented capabilities in information seeking and text generation, but challenges remain regarding citation errors and generating information not present in the evidence (hallucination). |
| Approach: | They propose a framework to assess citation errors and hallucination using an explicit evaluation paradigm to formulate actionable natural language feedback. |
| Outcome: | The proposed approach improves correctness, fluency, and citation quality and reduces hallucinations in the results. |