Papers with MMHal-Bench
Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have conjectured that multimodal hallucination is due to the vision encoder failing to ground on the image properly. |
| Approach: | They propose a multimodal self-feedback guided revision model that leverages visual cues to generate feedback to its initial response based on the visual information provided by the vision encoder. |
| Outcome: | The proposed model reduces multimodal hallucination and outperforms previous models on MMHal-Bench, POPE, and GAVIE. |
VLFeedback: A Large-Scale AI Feedback Dataset for Large Vision-Language Models Alignment (2024.emnlp-main)
Copied to clipboard
Lei Li, Zhihui Xie, Mukai Li, Shunian Chen, Peiyi Wang, Liang Chen, Yazheng Yang, Benyou Wang, Lingpeng Kong, Qi Liu
| Challenge: | Large vision-language models (LVLMs) are evolving rapidly and require data with human supervision to achieve better alignment. |
| Approach: | They introduce VLFeedback, the first large-scale vision-language feedback dataset . they train Silkie, an LVLM fine-tuned via direct preference optimization . |
| Outcome: | The proposed model outperforms its base model in helpfulness, visual faithfulness, and safety metrics and exhibits enhanced resilience against red-teaming attacks. |
Vision-Language Introspection: Mitigating Overconfident Hallucinations in MLLMs via Interpretable Bi-Causal Steering (2026.acl-long)
Copied to clipboard
Shuliang Liu, Songbo Yang, Dong Fang, Sihang Jia, Yuqi Tang, Lingfeng Su, Ruoshui Peng, Yibo Yan, Xin Zou, Xuming Hu
| Challenge: | Existing approaches to overcome object hallucination are limited . Existing mitigations include costly retraining and a training-free inference framework . |
| Approach: | They propose a training-free inference framework that simulates a metacognitive self-correction process. |
| Outcome: | The proposed framework reduces object hallucination rates by 12.67% on MMHal-Bench and improves accuracy by 5.8% on POPE. |
Inject to Heal: Alleviating hallucination in LVLMs via Context Embedding Injection (2026.findings-acl)
Copied to clipboard
| Challenge: | a large vision-language model can generate hallucinations inconsistent with visual input . a lightweight method that embeds the last input token as a grounding signal reduces the likelihood of hallucinosity. |
| Approach: | They propose a training-free mitigation strategy that harnesses the hidden state of the last input token as a grounding signal to maintain visual fidelity throughout decoding and curb hallucinations. |
| Outcome: | The proposed method outperforms state-of-the-art methods on CHAIR, AMBER, and MMHal benchmarks. |