| Challenge: | Prior work has shown that sentiment is encoded linearly in LLM representations, but their ability to utilize this information remains fragile to prompt variations. |
| Approach: | They propose a simple inference-time intervention method that amplifies circuit features to compensate for insufficient activation. |
| Outcome: | The proposed method improves on a sentiment analysis circuit with sparse autoencoders and circuit-level analysis. |
Similar Papers
LLaMAs Have Feelings Too: Unveiling Sentiment and Emotion Representations in LLaMA Models Through Probing (2025.acl-long)
Copied to clipboard
Dario Di Palma, Alessandro De Bellis, Giovanni Servedio, Vito Walter Anelli, Fedelucio Narducci, Tommaso Di Noia
| Challenge: | Large Language Models (LLMs) have become central to NLP, demonstrating their ability to adapt to various tasks through prompting techniques. |
| Approach: | They probe the hidden layers of Large Language Models to identify where sentiment features are most represented and to assess how this affects sentiment analysis. |
| Outcome: | The proposed approach enables sentiment tasks to be performed with memory requirements reduced by an average of 57%. |
A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Sparse Autoencoders (SAEs) can disentangle complex features into more interpretable components. |
| Approach: | They propose to use Sparse Autoencoders to disentangle LLM features into more interpretable components. |
| Outcome: | The proposed method disentangles complex features into more interpretable components. |
Sentiment Analysis in the Era of Large Language Models: A Reality Check (2024.findings-naacl)
Copied to clipboard
| Challenge: | Sentiment analysis (SA) has been a long-standing research area in natural language processing. |
| Approach: | They propose a benchmark to evaluate LLMs' SA abilities and propose 'sentiEval' benchmark to be used for a more comprehensive evaluation. |
| Outcome: | The proposed benchmark outperforms small language models on 26 datasets on 13 tasks and compared them with LLMs trained on domain-specific datasets. |
How Do LLMs Generate Contrastive Sentiments? A Mechanistic Perspective (2026.eacl-long)
Copied to clipboard
| Challenge: | Despite extensive research, the mechanisms underlying LLMs' abilities remain poorly understood. |
| Approach: | They propose and validate a mechanistic intervention that transforms the sentiment of a text from positive to negative while making minimal edits. |
| Outcome: | The proposed intervention increases sentiment flip rate without sacrificing minimal changes to text content. |
Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Models (2026.acl-long)
Copied to clipboard
Ruixuan Deng, Xiaoyang Hu, Miles Gilberti, Shane Storks, Aman Taxali, Mike Angstadt, Chandra Sripada, Joyce Chai
| Challenge: | Recent work has focused on layerwise interpretations, lacking fine-grained interpretation of specific features and their interaction. |
| Approach: | They identify semantically coherent, context-consistent network components in large language models . they use sparse autoencoders to coactivate sparsity features from a handful of prompts . |
| Outcome: | The proposed model can capture concepts and relations more comprehensively than individual features while maintaining specificity. |
Deciphering Cultural Representations in Large Language Models via Sparse Autoencoders (2026.findings-acl)
Copied to clipboard
| Challenge: | Prior work has identified so-called cultural neurons, but individual neurons are often polysemous, conflating abstract cultural knowledge with surface-level lexical cues due to superposition. |
| Approach: | They apply Sparse Autoencoders to decompose LLM activations into sparse, interpretable feature representations that disentangle culturally selective features. |
| Outcome: | The proposed model disentangles culturally selective features from paraphrasing and task formats, indicating abstraction beyond lexical correlations. |
Prompting the Unknown: Understanding Response Uncertainty in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are widely used in decision-making across diverse domains. |
| Approach: | They propose a prompt-response concept model that explains the relationship between the amount of task-relevant information provided in the prompt and the LLM-generated response uncertainty by identifying four sources of response uncertainty. |
| Outcome: | The proposed model shows that the amount of information provided in the prompt influences the LLM-generated response uncertainty. |
Beyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on Quality (2026.findings-acl)
Copied to clipboard
Qipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu
| Challenge: | Existing studies on prompt engineering have focused on optimizing models for performance under stylistic perturbations. |
| Approach: | They conduct the first analysis of n-gram token-level mechanisms . they find that higher average performance is inherently associated with lower variance and greater stability. |
| Outcome: | The proposed model reduces the variance of the generated code by 40% . the proposed model is based on a large-scale dataset of 132,000 prompt variants . |
Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit impressive abilities in various domains such as text generation, instruction following, and reasoning. |
| Approach: | They propose a method to decompose the activations of Large Language Models into a sparse linear combination of SAE features. |
| Outcome: | The proposed method shows that some features are strongly related to specific languages, while others are unaffected by ablating them. |
You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments (2024.naacl-long)
Copied to clipboard
Bangzhao Shu, Lechen Zhang, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, David Jurgens
| Challenge: | Large Language Models (LLMs) are popular for research in social sciences . currently, prompting LLMs is insufficient to accurately and reliably capture model perceptions, and we discuss potential alternatives to improve this. |
| Approach: | They construct a dataset that contains 693 questions encompassing 39 different instruments of persona measurement on 115 persona axes and a set of questions containing minor variations. |
| Outcome: | The proposed model can generate answers and negate statements in a consistent and robust manner. |