Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms (2025.acl-long)
Copied to clipboard
| Challenge: | Recent research has explored the use of sparse autoencoders (SAE) to disentangle knowledge in high-dimensional spaces for steering. |
| Approach: | They propose a method that isolates and manipulates disentangled knowledge components to enhance safety by using sparse autoencoders to disentangle knowledge in high-dimensional spaces for steering. |
| Outcome: | The proposed method is able to isolate and manipulate disentangled knowledge components to enhance safety in large reasoning models. |
Similar Papers
Improving LLM Reasoning through Interpretable Role-Playing Steering (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for role-playing rely on prompt engineering, which lacks stability and interpretability. |
| Approach: | They propose a framework that extracts latent representations from role-play prompts and constructs a steering vector that can be injected into the model's residual stream with controllable intensity. |
| Outcome: | The proposed framework extracts latent representations from role-play prompts, selects the most relevant features based on activation patterns, and constructs a steering vector that can be injected into the model’s residual stream with controllable intensity. |
SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive capabilities in natural language understanding and generation, but controlling their behavior remains a challenge. |
| Approach: | They propose a supervised steering approach that operates in sparse, interpretable representation spaces. |
| Outcome: | The proposed approach achieves higher success rates with minimal degradation in generation quality compared to existing methods. |
Steering Safely or Off a Cliff? Rethinking Specificity and Robustness in Inference-Time Interventions (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing studies have shown that model steering can preserve fluency and unrelated abilities, but it fails to preserve robustness specificity. |
| Approach: | They propose a framework that distinguishes three dimensions of specificity: general, control, and robustness. |
| Outcome: | The proposed framework distinguishes three dimensions of specificity: general (preserving fluency and unrelated abilities), control (preserving related control properties), and robustness (preserving control properties under distribution shifts). |
Toward Efficient Sparse Autoencoder-Guided Steering for Improved In-Context Learning in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Sparse autoencoders (SAEs) have emerged as a powerful analytical tool in mechanistic interpretability for large language models (LLMs). |
| Approach: | They propose a novel approach that leverages SAEs to enhance the general in-context learning performance of large language models (LLMs). |
| Outcome: | The proposed method yields a 3.5% improvement across diverse text classification tasks and exhibits greater robustness to hyperparameter variations compared to standard steering approaches. |
Evaluating the Impact of SAE-based Language Steering on LLM Performance (2026.eacl-srw)
Copied to clipboard
| Challenge: | Recent advances in Sparse Autoencoders (SAEs) have revealed interpretable features within large language models (LLMs) however, the impact of SAE-based language steering on output quality and task performance remains unclear. |
| Approach: | They apply language-specific SAE feature steering to three LLMs from two model families and evaluate it on a translation task and a multilingual question-answering task. |
| Outcome: | The proposed approach outperforms prompting and language neuron-based steering on translation and multilingual question-answering tasks. |
Controllable LLM Reasoning via Sparse Autoencoder-Based Steering (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods struggle to control fine-grained reasoning strategies due to conceptual entanglement in LRMs’ hidden states. |
| Approach: | They propose to decompose strategy-entangled hidden states into a disentangled feature space by using Sparse Autoencoders to identify the few strategy-specific features from the vast pool of SAE features. |
| Outcome: | The proposed method outperforms existing methods by 15% in control effectiveness. |
Beyond Linear Steering: Unified Multi-Attribute Control for Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Empirical results show that K-Steering outperforms strong baselines in accurately steering multiple behaviors. |
| Approach: | They propose a method that trains a single classifier on hidden activations and computes intervention directions via gradients at inference time. |
| Outcome: | The proposed method outperforms strong baselines in steering multiple behaviors. |
Noisy Exemplars Make Large Language Models More Robust: A Domain-Agnostic Behavioral Analysis (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on the robustness of LLMs with few-shot prompting techniques are limited. |
| Approach: | They propose to test the robustness of LLMs in multi-hop reasoning tasks via domain-agnostic perturbations. |
| Outcome: | The proposed model is more sensitive to certain perturbations such as replacing words with synonyms and more robust to few-shot prompting methods. |
FineSteer: A Unified Framework for Fine-Grained Inference-Time Steering in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for inference-time steering fail to be effective, utility-preserving and training-efficient due to rigid, one-size-fits-all designs and limited adaptability. |
| Approach: | They propose a steering framework that decomposes inference-time steering into two stages . they propose 'conditional steering' mechanism that preserves model utility by avoiding unnecessary steering . a 'mixture-of-Steering-Experts' mechanism captures multimodal nature of desired steering behaviors . |
| Outcome: | The proposed framework outperforms the state-of-the-art methods on safety and truthfulness benchmarks. |
Position Engineering: Boosting Large Language Models through Positional Information Manipulation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated significant strides towards achieving artificial general intelligence. |
| Approach: | They propose a technique termed position engineering which alters the positional information in the prompt without modifying the text itself. |
| Outcome: | The proposed technique significantly improves on the baseline in retrieval-augmented generation and in-context learning scenarios. |