Suzanne Petryk, David Chan, Anish Kachinthaya, Haodi Zou, John Canny, Joseph Gonzalez, Trevor Darrell
| Challenge: | Existing metric for object hallucination, CHAIR, is limited to MS COCO objects and synonyms. |
| Approach: | They propose a new open-vocabulary metric, ALOHa, which leverages large language models to measure object hallucinations. |
| Outcome: | The proposed metric correctly identifies 13.6% more hallucinated objects than CHAIR on HAT and 30.8% more on nocaps. |
Similar Papers
Object Hallucination in Image Captioning (D18-1)
Copied to clipboard
| Challenge: | Existing image captioning metrics do not capture image relevance . current metrics only measure similarity to ground truth captions . |
| Approach: | They propose a new image relevance metric to evaluate captioning models with veridical visual labels and assess their rate of object hallucination. |
| Outcome: | The proposed metrics show that models with veridical visual labels have higher hallucination rates than models with lower hallucinosity. |
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) often hallucinate and produce captions that mention concepts that cannot be found in the image. |
| Approach: | They propose to add grounding objectives to captions that explicitly align image regions or objects to text spans to reduce hallucination. |
| Outcome: | The proposed evaluation protocol reduces the amount of hallucination in LVLMs by adding grounding objectives. |
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training (2023.eacl-main)
Copied to clipboard
| Challenge: | Large-scale vision-language pre-trained (VLP) models generate unfaithful or nonsensical texts given the source input, which is called hallucination. |
| Approach: | They propose a VLP loss-based model to mitigate object hallucination by decoupling VLP objectives and a token-level image-text alignment. |
| Outcome: | The proposed model reduces object hallucination by 17.4% on two benchmarks. |
Evaluating Object Hallucination in Large Vision-Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) suffer from object hallucinations, i.e., they tend to generate objects inconsistent with the target images in the descriptions. |
| Approach: | They propose to integrate powerful large vision-language models (LVLMs) they propose a polling-based query method to evaluate object hallucination . |
| Outcome: | The proposed model can evaluate object hallucination in a more stable and flexible way. |
ANAH: Analytical Annotation of Hallucinations in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | a comprehensive and fine-grained measurement of the hallucination is crucial for LLMs' wide applications. |
| Approach: | They propose a dataset that offers ANalytical Annotation of Hallucinations in Large Language Models. |
| Outcome: | The proposed dataset can be used to train and evaluate hallucination annotators. |
Evaluating Evaluation Metrics – The Mirage of Hallucination Detection (2025.findings-emnlp)
Copied to clipboard
Atharva Kulkarni, Yuan Zhang, Joel Ruben Antony Moniz, Xiou Ge, Bo-Hsiang Tseng, Dhivya Piraviperumal, Swabha Swayamdipta, Hong Yu
| Challenge: | a large-scale empirical evaluation of hallucination detection metrics is conducted . hallucinosity is a significant obstacle to the reliability and widespread adoption of language models . |
| Approach: | They conduct large-scale empirical evaluation of hallucination detection metrics . they compare hallucinian language models, language models and decoding methods . |
| Outcome: | The results show that the evaluations of hallucination detection metrics fail to align with human judgments, they say . they also show that evaluations with LLM-based evaluation yield the best overall results . |
The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations (2023.emnlp-main)
Copied to clipboard
Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, Anubhav Sarkar, S.M Towhidul Islam Tonmoy, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Recent advances in Large Language Models have generated widespread acclaim, but hallucination has also emerged as a by-product. |
| Approach: | They propose a fine-grained discourse on profiling hallucination based on its degree, orientation, and category . they categorize hallucines into six types: acronym ambiguity, generated golem, virtual voice, geographic erratum, time wrap . |
| Outcome: | The proposed method categorizes hallucination into six types based on their degree, orientation, and category . |
InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for detecting hallucinations in large language models are limited due to their high frequency and high accuracy. |
| Approach: | They propose a method to detect hallucinations in large language models by repeating model-generated responses from its generated answer. |
| Outcome: | The proposed method achieves 87% hallucinations in a specific experiment without external knowledge. |
HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing studies on hallucination focus on text or vision, while few audio-oriented studies are limited in scale, modality coverage, and diagnostic depth. |
| Approach: | They propose a large-scale benchmark for evaluating hallucinations across speech, sound, and music. |
| Outcome: | The proposed model improves hallucination rate, yes/no bias, error-type analysis, and refusal rate. |
HAT: Hallucination Annotation for Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Hallucinations in machine translation (MT) outputs are prone to hallucination, authors say . lack of high-quality benchmarks for halluciation detection has hindered MT deployments . |
| Approach: | They propose a dataset that provides annotated hallucination distributions and benchmarks . they use 350,959 span-level annotations across 38 language pairs to analyze hallucis a MT output . |
| Outcome: | The proposed dataset provides high-quality benchmarks for hallucination detection in machine translation . the dataset includes 350,959 span-level annotated samples across 38 language pairs . |