K-GIP: Diagnosing Logical Fractures in Large Vision-Language Models via Verification Scene Graphs and Sequential Pruning (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks that treat hallucinations as isolated errors neglect causal dependencies between visual perception and textual reasoning. |
| Approach: | They propose a Knowledge-Guided In-Context Probing framework that constructs a dual-perception ground truth to transform abstract priors into multi-granularity queries. |
| Outcome: | The proposed framework isolates deep reasoning failures from simple perceptual misses. |
Similar Papers
Hidden in Plain Sight: Reasoning in Underspecified and Misspecified Scenarios for Multimodal LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal large language models are increasingly deployed in open-ended, real-world environments where inputs are messy, underspecified, and not always trustworthy. |
| Approach: | They evaluate multimodal large language models in real-world environments where inputs are messy, underspecified, and not always trustworthy. |
| Outcome: | The proposed models fail to detect hidden issues even when they possess the necessary perceptual and reasoning skills. |
KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions (2025.coling-main)
Copied to clipboard
| Challenge: | Existing benchmarks that assess this vulnerability rely on manual construction, resulting in limited size and lack of expandability. |
| Approach: | They propose a method to generate false premise questions based on knowledge graphs . they modify true triplets extracted from KGs to create false premises . |
| Outcome: | The proposed method generates semantically rich FPQs using state-of-the-art GPTs. |
VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Existing verbalized confidence calibration methods for large vision language models optimize a single holistic confidence score using binary answer-level correctness. |
| Approach: | They propose a reinforcement learning framework that explicitly decouples confidence into visual and reasoning confidence. |
| Outcome: | Experiments show that the proposed framework decouples confidence into visual and reasoning confidence while suppressing ungrounded hallucinations while preserving valid perception. |
FOL-Traces: Verified First-Order Logic Reasoning Traces at Scale (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing approaches to evaluate language models fail to provide structural clarity and verifiable inference. |
| Approach: | They propose to use a large-scale dataset of programmatically verified reasoning traces to evaluate structured logical inference. |
| Outcome: | The proposed model achieves 45.7% accuracy on masked operation prediction and 27% on two-step completion. |
Drivel-ology: Challenging LLMs with Interpreting Nonsense with Depth (2025.emnlp-main)
Copied to clipboard
| Challenge: | Despite excelling at many natural language processing tasks, large language models fail to grasp the layered semantics of Drivelological text. |
| Approach: | They construct a benchmark dataset of over 1,200+ carefully curated and diverse examples across English, Mandarin, Spanish, French, Japanese, and Korean to examine their Drivelological characteristics. |
| Outcome: | The proposed models lack conceptual understanding and lack conceptual and semantic accuracy. |
Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) have impressive capabilities in multi-modal context comprehension, but they still suffer from hallucination problems due to inconsistent outputs with the image content. |
| Approach: | They propose a training-free framework MVP to reduce hallucinations in Large Vision-Language Models . they propose multi-view information-seeking strategy to perceive the comprehensive information in the image . |
| Outcome: | The proposed framework reduces hallucinations in large vision-language models by combining multi-view multi-path reasoning with multi-vision multi-path reasoning. |
Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Object hallucination has been an Achilles’ heel which hinders the broader applications of large vision-language models (LVLMs). |
| Approach: | They propose a logical closed loop-based framework for Object Hallucination Detection and Mitigation that uses logical consistency probing to raise questions with logical correlations to determine hallucinations. |
| Outcome: | The proposed method can be applied to all existing LVLMs and is effective and general. |
Benchmarking Deflection and Hallucination in Large Vision-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks overlook conflicts between visual and textual evidence and the importance of generating deflections when incomplete knowledge is retrieved. |
| Approach: | They propose a dynamic curation pipeline that preserves benchmark difficulty over time . they propose 'vlm-DeflectionBench' benchmark to probe model behaviour under conflicting evidence . |
| Outcome: | The proposed benchmarks overlook conflicts between visual and textual evidence and are prone to obsolescence . the proposed benchmark is based on 2,775 samples spanning diverse retrieval settings . |
SceneAlign: Aligning Multimodal Reasoning to Scene Graphs in Complex Visual Scenes (2026.acl-long)
Copied to clipboard
Chuhan Wang, Xintong Li, Jennifer Yuntong Zhang, Junda Wu, Chengkai Huang, Lina Yao, Julian McAuley, Jingbo Shang
| Challenge: | Existing preference-based approaches fail to address this challenge by exploiting language priors to bypass visual grounding. |
| Approach: | They propose a framework that leverages scene graphs as structured visual information to perform controllable structural interventions. |
| Outcome: | The proposed framework improves answer accuracy and reasoning faithfulness across seven visual reasoning benchmarks. |
CHARPEVAL: Benchmarking Large Language Models’ Contextual Reasoning in Knowledge-Grounded Dialogue (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks that evaluate the ability of Large Language Models (LLMs) to perform contextualized reasoning in knowledge-grounded dialogue scenarios are lacking. |
| Approach: | They propose a benchmark to evaluate the ability of Large Language Models to perform contextualized reasoning in knowledge-grounded dialogue scenarios. |
| Outcome: | The proposed benchmark shows that open-weight LLMs are ineffective at reasoning over discontinuous chunks of text across the input. |