Viacheslav Sinii, Alexey Gorbatovski, Artem Cherepanov, Boris Shaposhnikov, Nikita Balagansky, Daniil Gavrilov
| Challenge: | Compared with LoRA and BitFit, training a single steering vector per layer with reinforcement learning requires orders of magnitude fewer resources and isolates a much smaller, more interpretable parameter set. |
| Approach: | They propose to train a single steering vector per layer with reinforcement learning while freezing all base weights to match the accuracy of fully RL-tuned reasoning models. |
| Outcome: | The proposed approach improves on an 8 billion-parameter model while keeping all base weights fixed. |
Similar Papers
CogSteer: Cognition-Inspired Selective Layer Intervention for Efficiently Steering Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) achieve excellent performance through pretraining on extensive data. |
| Approach: | They propose an efficient selective layer intervention based on parameter-efficient fine-tuning methods to select the optimal steering layer to modulate LLM semantics. |
| Outcome: | The proposed approach is based on a model-agnostic framework and is safe to deploy. |
Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement (2026.acl-long)
Copied to clipboard
Zexu Sun, Yongcheng Zeng, Erxue Min, Heyang Gao, Bokai Ji, Dugang Liu, Xing Tang, Xiuqiang He, Xu Chen
| Challenge: | Recent advances in Large Language Models have demonstrated notable inferential capacities via reinforcement learning (RL) however, “zero-RL” approaches relying on fixed prompt templates introduce substantial sampling inefficiencies for weak LLMs. |
| Approach: | They propose a hierarchical metacognitive RL framework that decomposes zero-accuracy problems into subproblems and prompts the policy to refine answers by referencing previous wrong solutions. |
| Outcome: | The proposed framework improves sample utilization and sample efficiency and accelerates convergence compared to baselines. |
ReFT: Reasoning with Reinforced Fine-Tuning (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve the generalization of large language models are using Supervised Fine-Tuning (SFT) this approach does not show sufficient generalization ability because it only relies on the given CoT data. |
| Approach: | They propose to use Chain-of-Thought annotations to train Large Language Models using supervised fine-tuning to improve generalization. |
| Outcome: | The proposed approach outperforms SFT on GSM8K, MathQA, and SVAMP datasets and shows a superior generalization ability. |
S2R: Teaching LLMs to Self-verify and Self-correct via Reinforcement Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to incentivize LLMs’ deep thinking abilities require large-scale data or significant training efforts. |
| Approach: | They introduce an efficient framework that enhances LLM reasoning by teaching models to self-verify and self-correct during inference. |
| Outcome: | The proposed framework outperforms models trained on long-CoT distilled data with 3.1k initialization samples and achieves an accuracy improvement of 51.0% to 81.6%. |
Exploring Chain-of-Thought Reasoning for Steerable Pluralistic Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are typically trained to reflect a relatively uniform set of values, which limits their applicability to tasks that require understanding of nuanced human perspectives. |
| Approach: | They propose to use Chain-of-Thought reasoning techniques to build steerable pluralistic models by fine-tuning on human-authored CoT and synthetic explanations. |
| Outcome: | The proposed methods outperform others and demonstrate strong sample efficiency. |
On the Impact of Fine-Tuning on Chain-of-Thought Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models have emerged as powerful tools for general intelligence, showcasing advanced natural language processing capabilities. |
| Approach: | They propose to use supervised fine-tuning and Quantized Low-Rank Adapters to improve LLMs' task-specific performance to address privacy and safety risks. |
| Outcome: | The proposed model improves the accuracy of the chain-of-thought reasonings across four datasets and demonstrates that the faithfulness of CoT reasoning decreases. |
Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMs (2026.findings-eacl)
Copied to clipboard
| Challenge: | Despite efforts to mitigate social bias in large language models, representational harms such as stereotyping continue to exist in both open and closed-source models. |
| Approach: | They propose a method to modify model activations in forward passes by applying steering vectors to a BBQ dataset and comparing their results to bias mitigation methods. |
| Outcome: | The proposed method outperforms 3 other bias mitigation methods on the BBQ dataset and shows the lowest impact on MMLU scores. |
RISER: Orchestrating Latent Reasoning Skills for Adaptive Activation Steering (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for domain-specific reasoning with large language models require updating parameter updates. |
| Approach: | They propose a plug-and-play intervention framework that adaptively steers LLM reasoning in activation space. |
| Outcome: | The proposed framework achieves zero-shot accuracy improvements of 3.4–6.5% over the base model while outperforming chain-of-thought-style reasoning with 2–3 higher token efficiency and robust accuracy gains. |
Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can solve reasoning and mathematical problems using the Chain-of-Thought technique, but require costly and long CoT data and fine-tuning. |
| Approach: | They propose a method that uses Sparse Autoencoders to extract interpretable features from vanilla CoT and use them to steer the LLM's internal states. |
| Outcome: | The proposed method uses Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT and steer the LLM's internal states during generation. |
Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly deployed in decision-making tasks where accuracy and reliable confidence estimates are essential. |
| Approach: | They propose a calibration-aware reinforcement learning formulation that directly adjusts decision-token probabilities. |
| Outcome: | The proposed model preserves RLVR’s accuracy level while mitigating overconfidence, reducing ECE scores up to 9 points. |