Small Language Models Need Strong Verifiers to Self-Correct Reasoning (2024.findings-acl)
Copied to clipboard
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, Lu Wang
| Challenge: | Existing studies show that large language models can self-correct their outputs by generating a critique and revising it based on the critique. |
| Approach: | They propose a pipeline that prompts small language models to collect self-correction data that supports the training of self-refinement abilities. |
| Outcome: | The proposed pipeline improves the self-correction abilities of two models on five datasets spanning math and commonsense reasoning. |
Similar Papers
Large Language Models Can Self-Improve (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have excellent performance in various tasks, but fine-tuning requires extensive supervision. |
| Approach: | They propose to use a pre-trained Large Language Model to generate rationale-augmented answers for unlabeled questions and fine-tune the LLM using those self-generated solutions as target outputs. |
| Outcome: | The proposed approach improves the general reasoning ability of a 540B-parameter LLM without any ground truth label. |
Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to align reasoning abilities between Large Language Models and Smaller Language Model are supervised fine-tuning and preference optimization. |
| Approach: | They propose a method that elicits Smaller Language Models to self-improve their reasoning abilities via preference optimization. |
| Outcome: | The proposed method outperforms Instruction-tuning on commonsense and math reasoning tasks on common and math scenarios. |
Self-Correction is More than Refinement: A Learning Framework for Visual and Language Reasoning Tasks (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable abilities, but they invariably generate flawed responses. |
| Approach: | They propose a self-correction approach that instructs VLMs to refine their outputs by allowing them to learn from their self-generated self-reference data without external feedback. |
| Outcome: | The proposed approach enables VLMs to learn from their self-generated self-correction data without relying on external feedback, facilitating self-improvement. |
Self-Correcting Code Generation Using Small Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has demonstrated that self-correction is a powerful tool for code generation, but whether it is effective for smaller models remains unexplored. |
| Approach: | They propose a method that trains small language models to maintain correct outputs while progressively correcting incorrect outputs as turns proceed. |
| Outcome: | The proposed approach improves the ability of small language models for multi-turn code correction. |
Self-Correction Makes LLMs Better Parsers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved remarkable success across various natural language processing tasks, but they still face challenges in performing fundamental NLP tasks, such as syntactic parsing. |
| Approach: | They propose a method that leverages grammar rules from existing treebanks to guide LLMs in correcting previous errors. |
| Outcome: | The proposed method significantly improves performance on in-domain and cross-domain datasets. |
Large Language Models are Better Reasoners with Self-Verification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to solve complex natural language processing tasks require multiple steps to verify the answers. |
| Approach: | They propose to use chain of thought prompting to solve reasoning tasks with large language models. |
| Outcome: | The proposed method can improve reasoning performance on arithmetic, commonsense, and logical reasoning datasets. |
LLMs cannot find reasoning errors, but can correct them given the error location (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent attempts to self-correct logical or reasoning errors often cause correct answers to become incorrect, resulting in poor performance overall. |
| Approach: | They propose to use a backtracking setup to test the correction abilities of LLMs on their mistake-finding ability to find logical mistakes. |
| Outcome: | The proposed model improves on 5 reasoning tasks, showing that it can correct logical mistakes without ground truth labels or training data. |
Teaching Language Models to Self-Improve through Interactive Demonstrations (2024.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have been shown to improve performance on downstream tasks by prompting them to analyze and revise their outputs. |
| Approach: | They propose a training algorithm that prompts large language models to analyze and revise their own outputs and uses this feedback to train the small model. |
| Outcome: | The proposed approach improves LLaMA-7B's performance on math and reasoning tasks by up to 7.13%. |
Small Language Models Improve Giants by Rewriting Their Outputs (2024.eacl-long)
Copied to clipboard
| Challenge: | despite impressive performance of large language models, they lag behind specialized models in various tasks. |
| Approach: | They propose a training model that can be integrated with different LLMs at inference to improve their performance without task-specific training. |
| Outcome: | The proposed model outperforms standard models on four natural language generation tasks. |
LightReasoner: Can Small Language Models Teach Large Language Models Reasoning? (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated remarkable progress in reasoning, but are resource-intensive and require large curated datasets. |
| Approach: | They propose a framework that leverages the behavioral divergence between a stronger expert model and a weaker amateur model. |
| Outcome: | The proposed framework improves accuracy by up to 28.1% while reducing time consumption by 90% and tuning token usage by 99%. |