Representation-based Broad Hallucination Detectors Fail to Generalize Out of Distribution (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a spurious correlation between hallucination detection methods and data is limiting the current SOTA. |
| Approach: | They propose a set of guidelines for hallucination detection and its evaluation. |
| Outcome: | The proposed method performs no better than supervised linear probes on the RAGTruth dataset . |
Similar Papers
Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks lack long context and label noise for stress-testing detectors . a new RAG-based HDB that underwent a rigorous human annotation process is developed . |
| Approach: | They propose a desiderata of properties for hallucination detection benchmarks to exhibit . they build a RAG-based HDB that underwent a rigorous human annotation process . |
| Outcome: | The proposed benchmark exhibits all desirable properties of existing HDBs . existing benchmarks lack realistic label noise for stress-testing detectors despite human annotation . |
Hallucination Detection in Long-Form Text Generated by LLMs: A Benchmark and a Hyper-Relational Knowledge Graph Approach (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection are coarse-grained and lack long-range consistency checks. |
| Approach: | They propose a benchmark for long-form hallucination detection that incorporates diverse entity types and intricate factual dependencies spanning extended contexts. |
| Outcome: | The proposed framework outperforms baselines and robustly integrates fact-centric hyper-relational knowledge graphs. |
The Impact of Negated Text on Hallucination with Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies on hallucination in large language models (LLMs) have been actively progressing in natural language processing. |
| Approach: | They propose to examine whether LLMs can recognize contextual shifts caused by negation and still reliably distinguish hallucinations comparable to affirmative cases. |
| Outcome: | The proposed model can detect hallucinations comparable to affirmative cases, but it is difficult to detect them in negated text, the authors show . |
Hallucination Detection for Grounded Instruction Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for generating instructions for navigation generate references to objects or actions that are inconsistent with what a human follower would perform or encounter along the path. |
| Approach: | They propose a weakly supervised approach that detects hallucinated references by using a pre-trained vision-language model. |
| Outcome: | The proposed model outperforms baseline models and supervised models on generating navigation instructions. |
The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (2023.emnlp-main)
Copied to clipboard
Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Recent advances in Large Language Models have generated widespread acclaim, but hallucination has also emerged as a by-product. |
| Approach: | They propose a fine-grained discourse on profiling hallucination based on its degree, orientation, and category . they categorize hallucines into six types: acronym ambiguity, generated golem, virtual voice, geographic erratum, time wrap . |
| Outcome: | The proposed method categorizes hallucination into six types based on their degree, orientation, and category . |
Enhancing Hallucination Detection through Perturbation-Based Synthetic Data Generation in System Responses (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection are expensive and outdated . despite the popularity of LLMs, the issue of hallucinosity poses significant concerns for downstream users. |
| Approach: | They propose an approach that automatically generates both faithful and hallucinated outputs by rewriting system responses. |
| Outcome: | The proposed model outperforms state-of-the-art zero-shot detectors and existing synthetic generation methods in accuracy and latency. |
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing work on pre-trained generative models often fails to detect non-existent or incorrect content . Existing studies have attempted to detect hallucinations based on oracle references . |
| Approach: | They propose a token-level, reference-free hallucination detection task based on Wikipedia annotations to detect non-existent or incorrect content. |
| Outcome: | The proposed task is token-level, reference-free hallucination detection task and dataset . authors argue that the proposed task can be used in real-time to detect hallucines . |
When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQA (2025.findings-emnlp)
Copied to clipboard
Elisei Rykov, Kseniia Petrushina, Maksim Savkin, Valerii Olisov, Artem Vazhentsev, Kseniia Titova, Alexander Panchenko, Vasily Konovalov, Julia Belikova
| Challenge: | Existing hallucination detection benchmarks operate at the sequence level and are limited to English . Existing methods lacking fine-grained, multilingual supervision are limited in English based on the sequence . |
| Approach: | They propose a large-scale, multilingual dataset annotated with span-level hallucinations across 14 languages. |
| Outcome: | The proposed dataset annotated with span-level hallucinations across 14 languages is scalable and cost-efficient. |
Embedding and Gradient Say Wrong: A White-Box Method for Hallucination Detection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection have attracted more attention from the community. |
| Approach: | They propose to model the distributional distance between the regular conditional output and the unconditional output, which is generated without a given input text. |
| Outcome: | The proposed model achieves state-of-the-art on the hallucination benchmarks HADES and other datasets. |
Evaluating Evaluation Metrics – The Mirage of Hallucination Detection (2025.findings-emnlp)
Copied to clipboard
Atharva Kulkarni, Yuan Zhang, Joel Ruben Antony Moniz, Xiou Ge, Bo-Hsiang Tseng, Dhivya Piraviperumal, Swabha Swayamdipta, Hong Yu
| Challenge: | a large-scale empirical evaluation of hallucination detection metrics is conducted . hallucinosity is a significant obstacle to the reliability and widespread adoption of language models . |
| Approach: | They conduct large-scale empirical evaluation of hallucination detection metrics . they compare hallucinian language models, language models and decoding methods . |
| Outcome: | The results show that the evaluations of hallucination detection metrics fail to align with human judgments, they say . they also show that evaluations with LLM-based evaluation yield the best overall results . |