Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows how to prompt large language models with explanations to obtain strong performance on textual reasoning tasks. |
| Approach: | They propose to optimize explanation-infused prompts in a blackbox fashion by using leave-one-out schemes and a two-stage framework. |
| Outcome: | The proposed method improves prompts over crowdworker annotations and naive search strategies. |
Similar Papers
Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues} (2023.findings-eacl)
Copied to clipboard
| Challenge: | Explanation prompts are used to generate an explanation for a given input . they are also used to improve model performance on adversarial benchmarks . |
| Approach: | They propose to use explanation prompts to generate an explanation that supports a label . they argue that prompting for explanations weakens superficial cues . |
| Outcome: | The proposed explanation prompts outperform label-only prompts on adversarial benchmarks. |
How Interpretable are Reasoning Explanations from Prompting Large Language Models? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prompt Engineering has garnered significant attention for enhancing the performance of large language models across a multitude of tasks. |
| Approach: | They propose a simple prompting technique that yields more than 70% improvement in interpretability. |
| Outcome: | The proposed method improves interpretability by 70% across multiple dimensions. |
Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting is a technique to enhance the reasoning abilities of Large language models (LLMs) however, the reasoning chains of demonstrations are observed to be prone to errors, which can lead to incorrect reasoning during inference. |
| Approach: | They propose an iterative bootstrapping technique to enhance the reasoning abilities of Large language models (LLMs) by generating a series of reasoning steps to obtain the answer, and using the reasoning chains as exemplars to demonstrate the task. |
| Outcome: | The proposed method improves the performance of Large language models (LLMs) on three reasoning tasks on ten datasets. |
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023.acl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). |
| Approach: | They propose to use Chain-of-Thought (CoT) prompting to encourage the LLM to generate intermediate rationales for solving a problem by providing a series of reasoning steps in the demonstrations. |
| Outcome: | The proposed model can generate coherent lines of reasoning even with invalid demonstrations while still generating coherent lines during inference. |
Explainable Inference Over Grounding-Abstract Chains for Science Questions (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing inference models for science questions are black-box by nature, lacking explanations for their predictions. |
| Approach: | They propose an explainable inference approach for science questions by reasoning on grounding and abstract inference chains. |
| Outcome: | The proposed model generates plausible explanations for science questions using a weighted graph of relevant facts and a Bayesian Optimisation formalism. |
Reframing Human-AI Collaboration for Generating Free-Text Explanations (2022.naacl-main)
Copied to clipboard
| Challenge: | Large language models are capable of generating fluent-appearing text with little task-specific supervision. |
| Approach: | They propose a pipeline that combines GPT-3 with a supervised filter that incorporates binary acceptability judgments from humans in the loop. |
| Outcome: | The proposed model can generate freetext explanations in a fewshot setting with human-written examples. |
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CoT synthesis approaches focus on simpler reasoning tasks and result in inconsistent CoT prompts. |
| Approach: | They propose a framework for automatic generation of superior CoT prompts based on three major evolution strategies . they propose 'step-level debating' method where multiple debaters discuss each reasoning step to arrive at the correct answer. |
| Outcome: | The proposed framework can generate superior CoT prompts from a CoT dataset. |
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains. |
| Approach: | They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple. |
| Outcome: | The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness. |
Threading the Needle: Reweaving Chain-of-Thought Reasoning to Explain Human Label Variation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have shown the power of chain-of-thought reasoning in improving complex decision-making tasks. |
| Approach: | They propose a pipeline that generates chain-of-thought (CoT) explanations from CoTs with improved accuracy. |
| Outcome: | The proposed pipeline outperforms a direct generation method and baselines on three datasets. |
Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a number of shortcomings, including lack of factual correctness. |
| Approach: | They propose a framework to increase prediction factuality by post-editing reasoning chains . they propose to use large language models to generate interpretable reasoning chains. |
| Outcome: | The proposed framework leads to accuracy improvements in open-domain question-answering tasks. |