ETF: An Entity Tracing Framework for Hallucination Detection in Code Summaries

Challenge:	Recent advances in large language models have significantly enhanced their ability to understand both natural language and code, but are prone to hallucinations.
Approach:	They propose a first-of-its-kind dataset, CodeSumEval, with 10K samples, curated specifically for hallucination detection in code summarisation.
Outcome:	The proposed framework has a 73% F1 score and is curated specifically for detection of hallucinations in code summarisation.

Similar Papers

Overview

Challenge

Approach

Outcome

Efficient Hallucination Detection in Automatic Code Generation (2026.findings-acl)

Copied to clipboard

Georgii Andriushchenko, Roman Garaev, Lyudmila Rvanova, Artem Shelmanov, Vladimir V. Ivanov

Responsible & Trustworthy NLP

Challenge:	Large language models produce source code that appears correct and well-formed, but includes hallucinated elements that cause downstream test failures.
Approach:	They develop a transformer-based detector that uses LLM internal representations to identify hallucinations.
Outcome:	The proposed detector outperforms existing methods and unsupervised methods in the code generation domain.

Tutorial Proposal: Hallucination in Large Language Models (2024.lrec-tutorials)

Copied to clipboard

Vipula Rawte, Aman Chadha, Amit Sheth, Amitava Das

Language Models

Challenge:	Grasping the intricacies of hallucination in LLMs can be daunting, especially for those new to the field.
Approach:	This tutorial aims to bridge the gap between the field and the field of hallucination . it will explore the key aspects of hallucinonation, including benchmarking, detection, and mitigation techniques .
Outcome:	This tutorial will explore the key aspects of hallucination in LLMs . it will also explore the specific constraints and shortcomings of current approaches .

Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps (2024.emnlp-main)

Copied to clipboard

Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, Ranjay Krishna, Yoon Kim, James Glass

Information Retrieval

Challenge:	Despite the utility and impressive capabilities of large language models, their tendency to generate hallucinations presents a significant challenge in their deployment.
Approach:	They propose a simple hallucination detection model based on the ratio of attention weights on the context versus newly generated tokens.
Outcome:	The proposed model reduces the amount of hallucinations by 9.6% in a summarization task.

HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023.emnlp-main)

Copied to clipboard

Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Natural Language Interfaces

Challenge:	Existing large language models (LLMs) are prone to generate hallucinations . a recent study shows that LLMs are able to generate content that conflicts with the source or cannot be verified by factual knowledge.
Approach:	They propose a framework to evaluate the performance of large language models (LLMs) they propose to use a sample of generated and human-annotated hallucinated samples to evaluate their performance .
Outcome:	The proposed framework generates and annotates hallucinated samples from ChatGPT . the results show that existing LLMs face great challenges in recognizing hallucines .

Towards Long Context Hallucination Detection (2025.findings-naacl)

Copied to clipboard

Siyi Liu, Kishaloy Halder, Zheng Qi, Wei Xiao, Nikolaos Pappas, Phu Mon Htut, Neha Anna John, Yassine Benajiba, Dan Roth

Reasoning

Challenge:	Large language models are prone to contextual hallucination, generating information that is either unsubstantiated or contradictory to the given context.
Approach:	They propose a dataset specifically designed for long-context hallucination detection.
Outcome:	The proposed architecture outperforms existing models while providing faster inference.

ANAH: Analytical Annotation of Hallucinations in Large Language Models (2024.acl-long)

Copied to clipboard

Ziwei Ji, Yuzhe Gu, Wenwei Zhang, Chengqi Lyu, Dahua Lin, Kai Chen

Language Models

Challenge:	a comprehensive and fine-grained measurement of the hallucination is crucial for LLMs' wide applications.
Approach:	They propose a dataset that offers ANalytical Annotation of Hallucinations in Large Language Models.
Outcome:	The proposed dataset can be used to train and evaluate hallucination annotators.

The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (2023.emnlp-main)

Copied to clipboard

Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, Amitava Das

Language Models

Challenge:	Recent advances in Large Language Models have generated widespread acclaim, but hallucination has also emerged as a by-product.
Approach:	They propose a fine-grained discourse on profiling hallucination based on its degree, orientation, and category . they categorize hallucines into six types: acronym ambiguity, generated golem, virtual voice, geographic erratum, time wrap .
Outcome:	The proposed method categorizes hallucination into six types based on their degree, orientation, and category .

Hallucination Detection in Long-Form Text Generated by LLMs: A Benchmark and a Hyper-Relational Knowledge Graph Approach (2026.findings-acl)

Copied to clipboard

Zituo Li, Guangzhou Chen, Jianbin Sun, Qi Song, Yang Kewei

Multimodality

Challenge:	Existing methods for hallucination detection are coarse-grained and lack long-range consistency checks.
Approach:	They propose a benchmark for long-form hallucination detection that incorporates diverse entity types and intricate factual dependencies spanning extended contexts.
Outcome:	The proposed framework outperforms baselines and robustly integrates fact-centric hyper-relational knowledge graphs.

The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models (2024.acl-long)

Copied to clipboard

Junyi Li, Jie Chen, Ruiyang Ren, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, Ji-Rong Wen

Language Models

Challenge:	a growing number of researchers are studying the hallucination issue in large language models.
Approach:	They propose a hallucination detection benchmark and a method to detect hallucines in LLMs.
Outcome:	The proposed method detects hallucinations and mitigates them using different training stages.

From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization (2025.findings-naacl)

Copied to clipboard

Catarina G Belém, Pouya Pezeshkpour, Hayate Iso, Seiji Maekawa, Nikita Bhutani, Estevam Hruschka

Text Generation

Challenge:	a recent study investigated hallucinations in multi-document summarization tasks . but, it is unclear how challenges arising from handling multiple documents affect outputs .
Approach:	They investigate how hallucinations manifest in large language models when summarizing topic-specific information from a set of documents.
Outcome:	The proposed benchmarks show that the models generate more hallucinations than baselines . the results highlight the need for more effective approaches to mitigate hallucinosity in MDS .