Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing research focuses on object-level or attribute-level hallucinations, neglecting the more complex relation hallucinosities. |
| Approach: | They propose a comprehensive benchmark targeting relation hallucinations comprising over 20,000 real-world samples and a confidence-based mitigation strategy which reduces the halluciation rate by an average of 9.75% across three datasets. |
| Outcome: | The proposed approach reduces the hallucination rate by an average of 9.75% across three datasets, including Reefknot. |
Similar Papers
RRHF-V: Ranking Responses to Mitigate Hallucinations in Multimodal Large Language Models with Human Feedback (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to mitigate hallucinations generate erroneous or fabricated information. |
| Approach: | They propose a rank-response-based model that annotates pair-reponses and trains alignment algorithms to improve the correspondence between images and text. |
| Outcome: | The proposed model outperforms the DPO method and outperfies existing methods on two MLLMs of different sizes and four widely used benchmarks. |
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing large language models (LLMs) are prone to generate hallucinations . a recent study shows that LLMs are able to generate content that conflicts with the source or cannot be verified by factual knowledge. |
| Approach: | They propose a framework to evaluate the performance of large language models (LLMs) they propose to use a sample of generated and human-annotated hallucinated samples to evaluate their performance . |
| Outcome: | The proposed framework generates and annotates hallucinated samples from ChatGPT . the results show that existing LLMs face great challenges in recognizing hallucines . |
Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) are known to hallucinate, which limits their practical applications. |
| Approach: | They propose a method that uses three types of preference pairs to target hallucinations from their diverse forms and causes. |
| Outcome: | The proposed method surpasses most state-of-the-art methods and shows potential for further improvements. |
Unified Hallucination Detection for Multimodal Large Language Models (2024.acl-long)
Copied to clipboard
Xiang Chen, Chenxi Wang, Yida Xue, Ningyu Zhang, Xiaoyan Yang, Qiang Li, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen
| Challenge: | despite significant strides in multimodal tasks, MLLMs are plagued by the critical issue of hallucination. |
| Approach: | They propose a meta-evaluation benchmark to facilitate evaluation of advancements in hallucination detection methods. |
| Outcome: | The proposed framework validates hallucinations robustly and provides strategic insights . MHaluBench is a meta-evaluation benchmark designed to facilitate evaluation . |
The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (2023.emnlp-main)
Copied to clipboard
Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Recent advances in Large Language Models have generated widespread acclaim, but hallucination has also emerged as a by-product. |
| Approach: | They propose a fine-grained discourse on profiling hallucination based on its degree, orientation, and category . they categorize hallucines into six types: acronym ambiguity, generated golem, virtual voice, geographic erratum, time wrap . |
| Outcome: | The proposed method categorizes hallucination into six types based on their degree, orientation, and category . |
EGOILLUSION: Benchmarking Hallucinations in Egocentric Video Understanding (2025.emnlp-main)
Copied to clipboard
Ashish Seth, Utkarsh Tyagi, Ramaneswaran Selvakumar, Nishit Anand, Sonal Kumar, Sreyan Ghosh, Ramani Duraiswami, Chirag Agarwal, Dinesh Manocha
| Challenge: | Multimodal Large Language Models excel at visual perception and reasoning in third-person and egocentric videos, but are prone to hallucinations, generating coherent yet inaccurate responses. |
| Approach: | They propose to use a benchmark to evaluate MLLM hallucinations in egocentric videos. |
| Outcome: | EGOILLUSION comprises 1,400 videos paired with 8,000 human-annotated open and closed-ended questions designed to trigger hallucinations in both visual and auditory cues in egocentric videos. |
HalluLens: LLM Hallucination Benchmark (2025.acl-long)
Copied to clipboard
Yejin Bang, Ziwei Ji, Alan Schelten, Anthony Hartshorn, Tara Fowler, Cheng Zhang, Nicola Cancedda, Pascale Fung
| Challenge: | Large language models (LLMs) generate responses that deviate from user input or training data, a phenomenon known as "hallucination" . |
| Approach: | They propose a hallucination benchmark HalluLens that includes both extrinsic and intrinsic evaluation tasks to distinguish between extrindic and intrinsic hallucines. |
| Outcome: | The proposed framework disentangles LLM hallucination from "factuality" and distinguishes between extrinsic and intrinsic hallucines to promote consistency and facilitate research. |
Hallucination Detection in Long-Form Text Generated by LLMs: A Benchmark and a Hyper-Relational Knowledge Graph Approach (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for hallucination detection are coarse-grained and lack long-range consistency checks. |
| Approach: | They propose a benchmark for long-form hallucination detection that incorporates diverse entity types and intricate factual dependencies spanning extended contexts. |
| Outcome: | The proposed framework outperforms baselines and robustly integrates fact-centric hyper-relational knowledge graphs. |
HalluDetect: Detecting, Mitigating, and Benchmarking Hallucinations in Conversational Systems in the Legal Domain (2025.emnlp-industry)
Copied to clipboard
Spandan Anaokar, Shrey Ganatra, Swapnil Bhattacharyya, Harshvivek Kashid, Shruthi N Nair, Reshma Sekhar, Siddharth Manohar, Rahul Hemrajani, Pushpak Bhattacharyya
| Challenge: | Large Language Models (LLMs) are widely used in industry but still produce hallucinations, limiting their reliability in critical applications. |
| Approach: | They propose to reduce hallucinations in consumer grievance chatbots by reducing their token accuracy by 0.4159 per turn. |
| Outcome: | The proposed system achieves an F1 score of 68.92% outperforming baseline detectors by 22.47% while maintaining the highest token accuracy. |
KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models are limited by static and narrow questions, leading to limited coverage and misleading evaluations. |
| Approach: | They propose a Knowledge Graph-based hallucination benchmark that assesses Large Language Models across the breadth and depth of their knowledge and provides a fairer and more comprehensive insight into LLM truthfulness. |
| Outcome: | The proposed framework assesses LLMs across breadth and depth of their knowledge, and provides a fairer and more comprehensive insight into LLM truthfulness. |