Revisit What You See: Revealing Visual Semantics in Vision Tokens to Guide LVLM Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Large Vision–Language Models (LVLMs) integrate visual perception with language understanding, but how vision information contributes to the model’s decoding process remains under-explored. |
| Approach: | They propose a simple training-free decoding method that guides text generation in Large Vision–Language Models by Referencing Vision Tokens. |
| Outcome: | The proposed method leverages the semantic information embedded within vision tokens by projecting it into the text token distribution. |
Similar Papers
Don’t Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Vision Language Models suffer from hallucinations, attributing incorrect or misleading features to images. |
| Approach: | They propose a test-time approach that recalibrates the influence of blind tokens . they identify blind token by analyzing layer-wise attention distributions over image tokens. |
| Outcome: | The proposed approach reduces hallucinations in large vision language models . it uses a contrastive decoding strategy to balance the influence of blind tokens . |
Reducing Token Redundancy in LVLMs: A Systematic Review of Token Pruning Methods (2026.acl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) excel at visual understanding but face severe computational bottlenecks when processing high-resolution images and long videos due to massive visual token counts. |
| Approach: | They propose a taxonomy categorizing methods into vision-side, LLM-side and hybrid paradigms and analyze token selection mechanisms and pruning strategy. |
| Outcome: | The proposed method selectively removes less informative tokens while maintaining performance. |
Autoregressive Semantic Visual Reconstruction Helps VLMs Understand Better (2026.findings-acl)
Copied to clipboard
| Challenge: | Typical large vision-language models emphasize vision-to-language alignment while overlooking fine-grained visual information. |
| Approach: | They introduce autoregressive semantic visual reconstruction (ASVR) that enables joint learning of visual and textual modalities within a unified autoregression framework. |
| Outcome: | The proposed model improves baselines and multimodal understanding benchmarks by 2-3%. |
Unveiling the Response of Large Vision-Language Models to Visually Absent Tokens (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) generate contextually relevant responses by jointly interpreting visual and textual inputs. |
| Approach: | They propose a method to classify whether an input token is visually grounded by reinterpreting question prompts or replacing the detected absent tokens during generation. |
| Outcome: | The proposed method mitigates the models’ tendency to falsely presume the visual presence of text input and its generality across various LVLMs. |
Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects (2026.findings-acl)
Copied to clipboard
Jun Zhang, Yicheng Ji, Feiyang Ren, Yihang Li, Bowen Zeng, Zonghao Chen, Ke Chen, Lidan Shou, Gang Chen, Huan Li
| Challenge: | Large Vision-Language Models are hindered by a systemic efficiency barrier known as visual token dominance. |
| Approach: | They propose a systematic taxonomy of efficiency techniques structured around the inference lifecycle . they examine visual encoding, prefilling, and decoding to understand bottlenecks . |
| Outcome: | The proposed techniques reveal how upstream decisions dictate downstream bottlenecks . the proposed techniques include hybrid compression and modality-aware decoding . |
Through the Magnifying Glass: Adaptive Perception Magnification for Hallucination-Free VLM Decoding (2026.acl-long)
Copied to clipboard
| Challenge: | Existing vision-language models suffer from visual hallucination, where the generated responses contain inaccuracies that are not grounded in the visual input. |
| Approach: | They propose a visual decoding method that iteratively isolates relevant visual tokens based on attention and magnifies the corresponding regions. |
| Outcome: | The proposed method reduces language biases and amplifies weights of visual embedding during decoding, while still preserving strong reasoning capabilities. |
Watermarking for Factuality: Guiding Vision-Language Models Toward Truth via Tri-layer Contrastive Decoding (2025.findings-emnlp)
Copied to clipboard
Kyungryul Back, Seongbeom Park, Milim Kim, Mincheol Kwon, SangHyeok Lee, Hyunyoung Lee, Junhee Cho, Seunghyun Park, Jinkyu Kim
| Challenge: | Large Vision-Language Models (LVLMs) have shown promising results on multimodal tasks, but remain prone to hallucinations due to their reliance on a single modality or memorizing training data without properly grounding their outputs. |
| Approach: | They propose a training-free, tri-layer contrastive decoding with watermarking that uses a watermark-related question to identify a pivot layer and apply tri-layered contrastive coding to generate the final output. |
| Outcome: | The proposed method reduces hallucinations and generates more visually grounded responses. |
Fixing Semantic Blind Spots in Anchor Tokens of dMLLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive models (ARMs) are prone to hallucinations due to their sequential text generation and high latency. |
| Approach: | They propose a training-free decoding strategy that augments the attention key space with a static, distance-aware matrix to reduce the attention sink effect on semantic anchors. |
| Outcome: | The proposed method reduces the attention sink effect on semantic anchors while enhancing their ability to aggregate global visual information. |
Mind Your Special Tokens! On the Importance of Dedicated Sequence-End Tokens in Vision-Language Embedding Models (2026.eacl-short)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) are highly sensitive to end-of-input artifacts in fine-tuning and inference data, e.g., whether input sequences end with punctuation or newline characters. |
| Approach: | They propose to convert generative LVLMs into vision-language encoders via contrastive learning objectives and use supervised contrastive objectives to train them. |
| Outcome: | The proposed approach improves visual and text representations and improves retrieval and (semantic) similarity tasks. |
Semantically Comprehensive Token Pruning in LVLMs via Maximizing Concept Coverage (2026.acl-long)
Copied to clipboard
| Challenge: | Existing visual token pruning methods leverage simple metrics derived from human experience, such as attention or similarity, to rank and select tokens within a highly entangled feature space. |
| Approach: | They propose a novel visual token pruning method that uses a concept-driven paradigm to quantify the Marginal Semantic Gain of each token's contribution to uncovered concepts. |
| Outcome: | The proposed method outperforms state-of-the-art methods in a concept-driven model while maintaining semantic completeness. |