SAAS: Solving Ability Amplification Strategy for Enhanced Mathematical Reasoning in Large Language Models (2024.emnlp-industry)
Copied to clipboard
| Challenge: | Existing approaches to enhance mathematical reasoning and problem-solving abilities of Large Language Models (LLMs) despite their remarkable performance across domains, a notable challenge persists in the realm of mathematical reasoning. |
| Approach: | They propose a sequential learning approach that integrates the Chain-of-Thought and the Program-ofThough. |
| Outcome: | The proposed approach achieves state-of-the-art (SOTA) performance by integrating CoT and PoT learning. |
Similar Papers
AS-ES Learning: Towards efficient CoT learning in small models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to induce Chain-of-Thought (CoT) in LLMs are limited and do not consider the importance of efficiently utilizing existing CoT data. |
| Approach: | They propose a new training paradigm which exploits the inherent information in CoT for iterative generation. |
| Outcome: | The proposed training paradigm surpasses direct seq2seq training on CoT-extensive tasks without data augmentation or altering the model itself. |
Aligning Large and Small Language Models via Chain-of-Thought Reasoning (2024.eacl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting empowers Large Language Models to solve complex reasoning tasks in a step-wise manner. |
| Approach: | They propose a method for aligning and transferring reasoning abilities between larger and smaller Language Models by using CoT-Demonstrations. |
| Outcome: | The proposed method outperforms baselines on question-answering and mathematical reasoning benchmarks. |
Feature Extraction and Steering for Enhanced Chain-of-Thought Reasoning in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) can solve reasoning and mathematical problems using the Chain-of-Thought technique, but require costly and long CoT data and fine-tuning. |
| Approach: | They propose a method that uses Sparse Autoencoders to extract interpretable features from vanilla CoT and use them to steer the LLM's internal states. |
| Outcome: | The proposed method uses Sparse Autoencoders (SAEs) to extract interpretable features from vanilla CoT and steer the LLM's internal states during generation. |
MetaLadder: Ascending Mathematical Solution Quality via Analogical-Problem Reasoning Transfer (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Current paradigms generate CoT and answers directly for a given problem, diverging from human problem-solving strategies to some extent. |
| Approach: | They propose a framework that explicitly prompts LLMs to recall and reflect on meta-problems alongside their CoT solutions before addressing the target problem. |
| Outcome: | The proposed framework outperforms standard CoT-based methods on mathematical benchmarks and significantly improves their reasoning accuracy. |
CoT-RAG: Integrating Chain of Thought and Retrieval-Augmented Generation to Enhance Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
Feiyang Li, Peng Fang, Zhan Shi, Arijit Khan, Fang Wang, Weihao Wang, null Zhangxin-hw, Cui Yongjian
| Challenge: | Chain-of-thought reasoning has two key limitations: lack of reliability when solely relying on LLM-generated reasoning chains and interference from natural language reasoning steps with the models’ inference logic. |
| Approach: | They propose a chain-of-thought reasoning framework with three key designs to address these issues. |
| Outcome: | The proposed framework improves the performance of large language models on complex tasks by incorporating knowledge graphs and learnable knowledge case-aware RAG. |
MIND: From Passive Mimicry to Active Reasoning through Capability-Aware Multi-Perspective CoT Distillation (2026.acl-long)
Copied to clipboard
Jin Cui, Jiaqi Guo, Jiepeng Zhou, Ruixuan Yang, Jiayi Lu, Jiajun Xu, Jiangcheng Song, Boran Zhao, Pengju Ren
| Challenge: | Existing approaches restrict students to following a single golden rationale and treat different reasoning paths independently, causing suboptimal performance. |
| Approach: | They propose a capability-adaptive framework that transitions distillation from passive mimicry to active cognitive construction and employ a feedback-driven inertia calibration mechanism to align supervision with the student’s current adaptability. |
| Outcome: | Experiments show that the proposed framework achieves state-of-the-art performance on both in-distribution and out-of distribution benchmarks. |
Learning Multi-Step Reasoning by Solving Arithmetic Tasks (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies have demonstrated large LMs’ impressive performance in solving math problems, but such ability seems only to emerge from models with abundant parameters. |
| Approach: | They propose to continuously pre-train LMs with the capabilities of multi-step reasoning by continuously pretraining them on a synthetic dataset MsAT. |
| Outcome: | The proposed method improves LMs' multi-step reasoning abilities on four math word problem datasets. |
Self-Training Elicits Concise Reasoning in Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Chain-of-thought reasoning has enabled large language models to use additional computation through intermediate tokens to solve complex tasks, but current models often generate more tokens than necessary to accomplish the task, incurring extraneous inference costs. |
| Approach: | They propose to fine-tune models with self-generated concise reasoning paths obtained by best-of-N sampling and few-shot conditioning in task-specific settings to elicit concise reasoning. |
| Outcome: | The proposed method reduces output tokens by 30% on GSM8K and MATH while maintaining average accuracy. |
To Code or not to Code? Adaptive Tool Integration for Math Language Models via Expectation-Maximization (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing tools that integrate chain-of-thought reasoning and code execution lack metacognitive awareness to integrate tools. |
| Approach: | They propose a framework that synergizes structured exploration with off-policy RL optimization to create a cycle between metacognitive tool-use decisions and evolving capabilities. |
| Outcome: | The proposed framework improves over 11% on MATH500 and 9.4% on AIME without o1-like CoT. |
Critic-CoT: Boosting the Reasoning Abilities of Large Language Model via Chain-of-Thought Critic (2025.findings-acl)
Copied to clipboard
Xin Zheng, Jie Lou, Boxi Cao, Xueru Wen, Yuqiu Ji, Hongyu Lin, Yaojie Lu, Xianpei Han, Debing Zhang, Le Sun
| Challenge: | Existing approaches to improve the reasoning performance of large language models rely on intuitive instance-level feedback, which limits the reasoning capabilities. |
| Approach: | They propose a framework that pushes LLMs toward System-2-like critic capability by using a step-wise CoT reasoning paradigm and automatic construction of weak-supervision data without human annotation. |
| Outcome: | The proposed model significantly improves task-solving performance by filtering out invalid solutions or iterative refinement. |