Faithfulness-Aware Uncertainty Quantification for Fact-Checking the Output of Retrieval-Augmented Generation

Challenge:	Existing approaches to mitigating hallucinations conflate factuality with faithfulness to the retrieved evidence, incorrectly labeling factually correct statements as hallucinos . Existing methods to mitigate hallucinics rely on a lack of training data coverage, input ambiguity, and architectural constraints.
Approach:	They propose a method for hallucination detection in Large Language Models enhanced with knowledge retrieval based on faithfulness to the retrieved context.
Outcome:	The proposed method outperforms unsupervised UQ baselines, RAG-specific methods, and supervised classifiers across multiple tasks and LLMs.

Similar Papers

Overview

Challenge

Approach

Outcome

An Automatic Method to Estimate Correctness of RAG (2025.coling-industry)

Copied to clipboard

Chi Zhang, Vivek V. Datla, Aditya Shrivastava, Alfy Samuel, Zhiqi Huang, Anoop Kumar, Daben Liu

Challenge:	Existing methods to assess the correctness of RAG models fail to capture the model’s internal state during answer generation.
Approach:	They propose a method to predict the correctness of RAG models by modeling the model’s uncertainty on quantified perturbations of input.
Outcome:	Extensive experiments across multiple large language models show that the proposed approach quantifies RAG robustness by aligning predictions with ground truth with a MSE 0.002 while offering flexibility for diverse qualitative metrics.

Provenance: A Light-weight Fact-checker for Retrieval Augmented LLM Generation Output (2024.emnlp-industry)

Copied to clipboard

Hithesh Sankararaman, Mohammed Yasin, Tanner Sorensen, Alessandro Bari, Andreas Stolcke

Information Retrieval

Challenge:	Existing methods for fact checking RAG outputs rely on large language models.
Approach:	They propose a method that computes a factuality score that can be thresholded to yield a binary decision to check RAG outputs.
Outcome:	The proposed method is low latency and low cost at run-time and no need for LLM fine-tuning.

Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph (2026.findings-acl)

Copied to clipboard

Jianpeng Hu, Yanzeng Li, Jialun Zhong, Lei Zou, Wenfa Qi

Reasoning

Challenge:	Existing methods for detecting faithfulness hallucinations are coarse or do not capture the models’ internal reasoning processes, making it difficult to learn.
Approach:	They propose a semantic-level internal reasoning graph-based method for detecting faithfulness hallucination using Large language models.
Outcome:	The proposed method achieves better overall performance compared to state-of-the-art baselines on RAGTruth and Dolly-15k.

RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models (2024.acl-long)

Copied to clipboard

Cheng Niu, Yuanhao Wu, Juno Zhu, Siliang Xu, KaShun Shum, Randy Zhong, Juntong Song, Tong Zhang

Responsible & Trustworthy NLP

Challenge:	Retrieval-augmented generation (RAG) is a main technique for alleviating hallucinations in large language models.
Approach:	They propose to integrate RAG into large language models to analyze word-level hallucinations using a corpus of 18,000 naturally generated responses from diverse LLMs.
Outcome:	The proposed model can fine tune a relatively small LLM and achieve a competitive hallucination detection performance when compared to the existing prompt-based approaches.

Semantic Consistency-Based Uncertainty Quantification for Factuality in Radiology Report Generation (2025.findings-naacl)

Copied to clipboard

Chenyu Wang, Weichao Zhou, Shantanu Ghosh, Kayhan Batmanghelich, Wenchao Li

Reasoning

Challenge:	Radiology report generation has shown great potential in assisting radiologists . generative medical Vision Large Language Models (VLLMs) are prone to hallucinations and can produce inaccurate diagnostic information.
Approach:	They propose a framework that provides both report-level and sentence-level uncertainties.
Outcome:	The proposed method improves factuality scores by 10% by rejecting 20% of reports on the MIMIC-CXR dataset.

Adaptive Retrieval Without Self-Knowledge? Bringing Uncertainty Back Home (2025.acl-long)

Copied to clipboard

Viktor Moskvoretskii, Maria Marina, Mikhail Salnikov, Nikolay Ivanov, Sergey Pletenev, Daria Galimzianova, Nikita Krayko, Vasily Konovalov, Irina Nikishina, Alexander Panchenko

Question Answering

Challenge:	Recent adaptive retrieval methods integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge, but they often neglect efficiency evaluations and comparisons with uncertainty estimation techniques.
Approach:	They propose to integrate LLMs’ intrinsic knowledge with external information appealing to LLM self-knowledge but neglect efficiency evaluations and comparisons with uncertainty estimation techniques.
Outcome:	The proposed methods outperform complex pipelines in terms of efficiency and self-knowledge while maintaining comparable QA performance.

QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation (2026.findings-acl)

Copied to clipboard

Dehai Min, Kailin Zhang, Tongtong Wu, Lu Cheng

Reasoning

Challenge:	Existing methods for reducing LLM hallucinations rely on model-internal signals . Existing approaches rely only on model internal signals, resulting in unreliability .
Approach:	They propose a method that shifts from subjective confidence to objective statistics . they leverage Infini-gram for millisecond-latency queries over 4 trillion tokens .
Outcome:	The proposed method reduces hallucinations in large language models by reducing uncertainty in the model.

Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantification (2024.findings-acl)

Copied to clipboard

Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, Maxim Panov

Reasoning

Challenge:	Large language models are notorious for producing erroneous claims in their output.
Approach:	They propose a fact-checking and hallucination detection pipeline based on token-level uncertainty quantification that removes the impact of uncertainty about what claim to generate on the current step and what surface form to use.
Outcome:	The proposed method can fact-check the atomic claims in the output of large language models.

RAG-HAT: A Hallucination-Aware Tuning Pipeline for LLM in Retrieval-Augmented Generation (2024.emnlp-industry)

Copied to clipboard

Juntong Song, Xingguang Wang, Juno Zhu, Yuanhao Wu, Xuxin Cheng, Randy Zhong, Cheng Niu

Reasoning

Challenge:	Retrieval-augmented generation (RAG) has emerged as a significant advancement in the field of large language models (LLMs).
Approach:	They propose a method that uses hallucination detection labels to correct hallucines by integrating up-to-date information into their initial training.
Outcome:	The proposed method is based on the Retrieval Augmented Generation (RAG) method, which has shown to be effective in mitigating hallucinations and improving answer quality.

Stable-RAG: Mitigating Retrieval-Permutation-Induced Hallucinations in Retrieval-Augmented Generation (2026.acl-long)

Copied to clipboard

Qianchi Zhang, Hainan Zhang, Liang Pang, Hong-Wei Zheng, Zhiming Zheng

Responsible & Trustworthy NLP

Challenge:	Existing RAG methods focus on enhancing LLM robustness to low-quality retrieval, but neither address permutation sensitivity.
Approach:	They propose a method that exploits permutation sensitivity to mitigate hallucinations in Large Language Models.
Outcome:	The proposed model improves answer accuracy, reasoning consistency, and generalization across datasets, retrievers, and input lengths compared with strong baselines.