Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Autoregressive Large Language Models (LLMs) have demonstrated "emergent abilities" such as in-context learning, instruction following and reasoning. |
| Approach: | They propose a method that generates rationales from post hoc explanation methods applied to small language models to improve their own performance. |
| Outcome: | The proposed method improves on four SLMs and five datasets with strong reasoning abilities. |
Similar Papers
Using Natural Language Explanations to Improve Robustness of In-context Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that large language models excel in many tasks via in-context learning (ICL). However, ICL struggles to execute complex tasks such as arithmetic, commonsense, and symbolic reasoning. |
| Approach: | They propose to augment ICL with natural language explanations (NLEs) to produce further NLEs on adversarial datasets. |
| Outcome: | The proposed approach yields more accurate results than zero-shot-ICL and using only human-generated NLEs on eight adversarial datasets. |
Large Language Models Can Self-Improve (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have excellent performance in various tasks, but fine-tuning requires extensive supervision. |
| Approach: | They propose to use a pre-trained Large Language Model to generate rationale-augmented answers for unlabeled questions and fine-tune the LLM using those self-generated solutions as target outputs. |
| Outcome: | The proposed approach improves the general reasoning ability of a 540B-parameter LLM without any ground truth label. |
Self-Critique and Refinement for Faithful Natural Language Explanations (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work has demonstrated that Large Language Models (LLMs) can self-critique and refine their initial outputs, but this capability remains unexplored for improving explanation faithfulness. |
| Approach: | They propose a framework that enables models to improve the faithfulness of their own explanations through an iterative critique and refinement process without external supervision. |
| Outcome: | The proposed framework reduces unfaithfulness rates in three datasets and four state-of-the-art LLMs by 36% compared to 54.81% for baseline. |
Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning (2024.acl-long)
Copied to clipboard
Yue Yu, Jiaming Shen, Tianqi Liu, Zhen Qin, Jing Nathan Yan, Jialu Liu, Chao Zhang, Michael Bendersky
| Challenge: | Recent advances in natural language processing (NLP) have witnessed the remarkable capabilities of Large Language Models (LLMs). |
| Approach: | They propose an Explanation-Aware Soft Ensemble framework to empower in-context learning with Large language models. |
| Outcome: | The proposed framework can be used to enhance in-context learning on seven natural language understanding tasks and four varying-size LLMs. |
ZARA: Improving Few-Shot Self-Rationalization for Small Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies demonstrate great performance gain for self-rationalization by few-shot prompting LMs with rationale-augmented exemplars. |
| Approach: | They propose to leverage explanations for small LMs to improve few-shot self-rationalization by reducing the problem of plausibility judgement to natural language inference. |
| Outcome: | The proposed approach achieves SOTA performance on the FEB benchmark, for both the task accuracy and the explanation metric. |
LLMs Don’t Know Their Own Decision Boundaries: The Unreliability of Self-Generated Counterfactual Explanations (2025.emnlp-main)
Copied to clipboard
Harry Mayne, Ryan Othniel Kearns, Yushi Yang, Andrew M. Bean, Eoin D. Delaney, Chris Russell, Adam Mahdi
| Challenge: | Existing studies on language models' ability to explain their decisions in natural language have focused on self-generated counterfactual explanations (SCEs). |
| Approach: | They evaluate whether LLMs can generate valid counterfactuals and minimal ones . authors suggest that SCEs are, at best, an ineffective explainability tool . |
| Outcome: | The proposed language models can explain their decisions in natural language, the study finds . the models can produce valid counterfactual explanations, but make small edits that fail to change predictions. |
SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationales (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to elicit confidence from large language models are limited to binary or inaccurate group-level confidence estimates. |
| Approach: | They propose a training framework that teaches LLMs to express more fine-grained confidence estimates. |
| Outcome: | The proposed training framework reduces the confidence calibration error and maintains the performance of the model. |
Improving Language Model Reasoning with Self-motivated Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Large-scale high-quality training data is important for improving the performance of models. |
| Approach: | They propose a framework that motivates the model to automatically generate rationales on existing datasets and improves the performance of reasoning through reinforcement learning. |
| Outcome: | The proposed model outperforms InstructGPT on multiple reasoning datasets and outperformed InstructGPT on other datasets. |
Self-Refine Instruction-Tuning for Aligning Reasoning in Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to align reasoning abilities between Large Language Models and Smaller Language Model are supervised fine-tuning and preference optimization. |
| Approach: | They propose a method that elicits Smaller Language Models to self-improve their reasoning abilities via preference optimization. |
| Outcome: | The proposed method outperforms Instruction-tuning on commonsense and math reasoning tasks on common and math scenarios. |
SEK: Self-Explained Keywords Empower Large Language Models for Code Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have achieved impressive performance in code generation. |
| Approach: | They propose a technique that extracts and explicates the key terms in the problem description with the LLM itself. |
| Outcome: | The proposed technique improves the Pass@1 of DeepSeek-Coder-V2-Instruct from 85.4% to 93.3% on the humaneval benchmark. |