From Heads to Neurons: Causal Attribution and Steering in Multi-Task Vision–Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing models focus on single tasks, limiting comparability of neuron importance . ranking strategies overlook how task-dependent information pathways shape write-in effects of feed-forward network (FFN) neurons. |
| Approach: | They propose a gradient-free framework for task-aware neuron attribution and steering in multi-task vision-language models. |
| Outcome: | The proposed framework outperforms existing methods in identifying task-critical neurons and improves model performance after steering. |
Similar Papers
Neuron-Level Knowledge Attribution in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for attribution of knowledge in large language models struggle to operate at neuron level due to computational constraints. |
| Approach: | They propose a static method for pinpointing significant neurons using three metrics . they also propose identifying "query neurons" which activate these "value neurons" |
| Outcome: | The proposed method shows superior performance across three metrics compared to seven other methods . it analyzes six types of knowledge across attention and feed-forward network layers . |
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models often ignore token-level causal influence and underutilize model logits. |
| Approach: | They propose a novel approach that uses a gradient-based approach to identify influential tokens and construct directional steering vectors based on their contribution to preferred over dispreferred outputs. |
| Outcome: | The proposed approach outperforms fine-tuning and prior steering methods on both LLM and VLM tasks without degrading fluency or general capabilities. |
Constructing Interpretable Features from Compositional Neuron Groups (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for analyzing LLMs rely on dictionary learning with sparse autoencoders (SAEs) however, SAEs struggle in causal evaluations and lack intrinsic interpretability, as their learning is not explicitly tied to the computations of the model. |
| Approach: | They propose to decompose MLP activations with semi-nonnegative matrix factorization (SNMF) such that the learned features are mapped to their activating inputs, making them directly interpretable. |
| Outcome: | Experiments on Llama 3.1, Gemma 2 and GPT-2 show that SNMF derived features outperform SAEs and a strong supervised baseline on causal steering while aligning with human-interpretable concepts. |
Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence (2025.acl-long)
Copied to clipboard
Jinghan He, Kuan Zhu, Haiyun Guo, Junfeng Fang, Zhenglin Hua, Yuheng Jia, Ming Tang, Tat-Seng Chua, Jinqiao Wang
| Challenge: | Existing methods focus on alignment training or decoding refinements but address symptoms at the generation stage without probing the underlying causes. |
| Approach: | They propose a training-free approach to mitigate hallucination by enhancing the role of vision-aware attention heads. |
| Outcome: | The proposed method achieves superior performance compared to state-of-the-art approaches in mitigating hallucinations while maintaining high efficiency with negligible additional time overhead. |
Towards Understanding Multi-Task Learning (Generalization) of LLMs via Detecting and Exploring Task-Specific Neurons (2025.coling-main)
Copied to clipboard
| Challenge: | Despite their superior multitask capabilities, the multitask learning mechanisms of large language models remain as an open question. |
| Approach: | They propose a method that fine-tunes current task-specific neurons during continuous learning by using gradient attribution on task-specified data. |
| Outcome: | The proposed method is highly correlated with the given task and solves two common problems in multi-task learning and continuous learning: Generalization and Catastrophic Forgetting. |
Are the Values of LLMs Structurally Aligned with Humans? A Causal Perspective (2025.findings-acl)
Copied to clipboard
Yipeng Kang, Junqi Wang, Yexin Li, Mengmeng Wang, Wenming Tu, Quansen Wang, Hengli Li, Tingjun Wu, Xue Feng, Fangwei Zhong, Zilong Zheng
| Challenge: | Current approaches to value alignment focus on a few core values, such as helpfulness, harmlessness, and honesty. |
| Approach: | They propose to use latent causal value graphs to guide two lightweight value-steering methods . role-based prompting and sparse autoencoder (SAE) steering are also used . |
| Outcome: | Experiments on Gemma-2B-IT and Llama3-8B- IT show that the proposed methods are effective and controllable. |
Mitigating Hallucinations in VLMs: Enhancing Visual Attention via Head-Wise Perturbation (2026.findings-acl)
Copied to clipboard
| Challenge: | Vision–Language Models (VLMs) have demonstrated strong capabilities in tasks that require joint understanding of text and images. |
| Approach: | They propose a strategy that incorporates head-wise attention perturbation via continuous multiplicative noise coupled with a visual-guided loss focused on vision-sensitive text tokens to promote a more balanced attention distribution. |
| Outcome: | The proposed approach outperforms baseline models on three benchmarks and consistently outperformed the baseline model. |
LITE: Intent-based Task Representation Learning Using Weak Supervision (2022.naacl-main)
Copied to clipboard
| Challenge: | To-do texts are often short and under-specified, which poses a challenge for current text representation models. |
| Approach: | They propose a neural multi-task learning framework that extracts representations of English to-do tasks with a multi-head attention mechanism on top of a pre-trained text encoder. |
| Outcome: | The proposed model outperforms baseline models on four downstream tasks and achieves error reduction of 38.7%. |
Discovering and Causally Validating Emotion-Sensitive Neurons in Large Audio-Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Emotion is a central dimension of spoken communication, yet we lack a mechanistic account of how LALMs encode it internally. |
| Approach: | They propose to use emotion-sensitive neurons in large audio-language models to study their interpretations. |
| Outcome: | The proposed models show that they can be used to make decisions on emotion . the results show that the ESNs exhibit non-uniform clustering with partial cross-dataset transfer . |
Beyond Multiple Choice: Evaluating Steering Vectors for Summarization (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent methods for controlling language models can often be classified into three main strategies: prompt engineering, trainable decoding mechanisms, fine-tuning according to specific objectives. |
| Approach: | They evaluate steering vectors for controlling topical focus, sentiment, toxicity, and readability in abstractive summaries across the SAMSum, NEWTS, and arXiv datasets. |
| Outcome: | The proposed method is effective in free-form generation, but high steering strengths induce degenerate repetition and factual hallucinations. |