Mitigating Hallucinations in Multimodal Spatial Relations through Constraint-Aware Prompting (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing research has explored methods to enhance the performance of large vision-language models in spatial relations. |
| Approach: | They propose a constraint-aware prompting framework to reduce spatial relation hallucinations by incorporating two types of constraints into the prompt. |
| Outcome: | The proposed framework improves on three widely-used spatial relation datasets. |
Similar Papers
Mitigating Action-Relation Hallucinations in LVLMs via Relation-aware Visual Enhancement (2026.acl-long)
Copied to clipboard
| Challenge: | Existing research has focused on mitigating object hallucinations but often overlooks more complex relation hallucines, especially action relations involving interactions between objects. |
| Approach: | They propose a framework to locate action-relevant image regions and enhance the LVLM’s attention to those regions by using a Relation-aware Visual Enhancement method. |
| Outcome: | The proposed method achieves superior performance in mitigating action-relation hallucinations with negligible additional inference cost. |
Visual Evidence Prompting Mitigates Hallucinations in Large Vision-Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | LVLMs have shown impressive progress by integrating visual perception with linguistic understanding to produce contextually grounded outputs. |
| Approach: | They propose a visual evidence prompting method to mitigate hallucinations in large vision-language models by using small visual models to complement them. |
| Outcome: | The proposed method reduces hallucinations by reducing false activation and enhancing correct ones. |
SHARP: Steering Hallucination in LVLMs via Representation Engineering (2025.emnlp-main)
Copied to clipboard
Junfei Wu, Yue Ding, Guofan Liu, Tianze Xia, Ziyue Huang, Dianbo Sui, Qiang Liu, Shu Wu, Liang Wang, Tieniu Tan
| Challenge: | Large Vision-Language Models (LVLMs) generate responses that are plausible but incorrect or unsupported—commonly referred to as hallucinations. |
| Approach: | They propose a representation-level intervention framework that modulates hallucination-related features during inference by probing their encoded features. |
| Outcome: | The proposed framework reduces hallucinations while maintaining the performance and generalization capabilities of Large Vision-Language Models (LVLMs). |
Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models (2025.naacl-short)
Copied to clipboard
| Challenge: | Existing methods to mitigate object hallucination are impractical for proprietary LVLMs. |
| Approach: | They propose a framework to identify optimal visual prompts that enhance LVLM responses without access to model internals. |
| Outcome: | The proposed approach is model-agnostic and can be used on open-source and proprietary LVLMs. |
Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing research focuses on object-level or attribute-level hallucinations, neglecting the more complex relation hallucinosities. |
| Approach: | They propose a comprehensive benchmark targeting relation hallucinations comprising over 20,000 real-world samples and a confidence-based mitigation strategy which reduces the halluciation rate by an average of 9.75% across three datasets. |
| Outcome: | The proposed approach reduces the hallucination rate by an average of 9.75% across three datasets, including Reefknot. |
Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models (2024.acl-long)
Copied to clipboard
Weihong Zhong, Xiaocheng Feng, Liang Zhao, Qiming Li, Lei Huang, Yuxuan Gu, Weitao Ma, Yuan Xu, Bing Qin
| Challenge: | Large Vision-Language Models (LVLMs) suffer from multimodal hallucinations . however, the generated hallucines could influence the models’ subsequent generation . |
| Approach: | They propose a framework to evaluate LVLMs' behaviors when encountering generated hallucinations and a method to revise the output distribution of LVLs with the one derived from the residual visual input. |
| Outcome: | The proposed framework reduces the performance of open-source LVLMs by 31%, indicating that they are prone to accept the generated hallucinations and make false claims that they would not have supported without distractions. |
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large vision-language models (LVLMs) often hallucinate and produce captions that mention concepts that cannot be found in the image. |
| Approach: | They propose to add grounding objectives to captions that explicitly align image regions or objects to text spans to reduce hallucination. |
| Outcome: | The proposed evaluation protocol reduces the amount of hallucination in LVLMs by adding grounding objectives. |
Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering (2026.acl-long)
Copied to clipboard
Shuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu
| Challenge: | Existing approaches to overcome object hallucination are limited . Existing mitigations include costly retraining and a training-free inference framework . |
| Approach: | They propose a training-free inference framework that simulates a metacognitive self-correction process. |
| Outcome: | The proposed framework reduces object hallucination rates by 12.67% on MMHal-Bench and improves accuracy by 5.8% on POPE. |
HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing work detects hallucination by directly judging whether an object exists in an image, overlooking the association between the object and semantics. |
| Approach: | They propose a framework that incorporates hallucination feedback at both object and sentence semantic levels to alleviate over 15% of hallucinism. |
| Outcome: | The proposed framework can alleviate over 15% of hallucination even with a marginal degree of training. |
Mitigating Hallucinations in Large Vision-Language Models without Performance Degradation (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large vision-language models produce hallucinations that compromise output reliability. |
| Approach: | They propose a dual-stage framework for mitigating hallucinations without performance degradation . they propose semantic-aware component disentanglement and interpretable parameter updates . |
| Outcome: | The proposed model reduces hallucinations by 23.4% while maintaining 97.4% of general generative capability. |