R3 Prompting: Review, Rephrase and Resolve for Chain-of-Thought Reasoning in Large Language Models under Noisy Context (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have evaluated LLMs under noise-free context but the dilemma for LLM to produce inaccurate results under noisy context has not been fully investigated. |
| Approach: | They propose a new method for CoT reasoning using Chain-of-Thought prompting that interacts with LLMs to perform key sentence extraction, variable declaration and answer prediction. |
| Outcome: | The proposed method outperforms existing CoT prompting methods on five reasoning tasks under noisy context. |
Similar Papers
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023.acl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). |
| Approach: | They propose to use Chain-of-Thought (CoT) prompting to encourage the LLM to generate intermediate rationales for solving a problem by providing a series of reasoning steps in the demonstrations. |
| Outcome: | The proposed model can generate coherent lines of reasoning even with invalid demonstrations while still generating coherent lines during inference. |
Cue-CoT: Chain-of-thought Prompting for Responding to In-depth Dialogue Questions with LLMs (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing LLMs generate responses based on the dialogue context, overlooking the underlying linguistic cues about the user status exhibited in the context. |
| Approach: | They propose a linguistic cue-based chain-of-thoughts method which enhances the LLMs inference with an intermediate reasoning step to find cues exhibited in the dialogue. |
| Outcome: | The proposed method outperforms standard prompting methods on in-depth dialogue questions and linguistic cues exhibited in the context. |
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CoT synthesis approaches focus on simpler reasoning tasks and result in inconsistent CoT prompts. |
| Approach: | They propose a framework for automatic generation of superior CoT prompts based on three major evolution strategies . they propose 'step-level debating' method where multiple debaters discuss each reasoning step to arrive at the correct answer. |
| Outcome: | The proposed framework can generate superior CoT prompts from a CoT dataset. |
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains. |
| Approach: | They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple. |
| Outcome: | The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness. |
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is popular in reasoning tasks, but its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. |
| Approach: | They propose a Coarse-to-Fine Chain-of-Thought approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire essential concepts. |
| Outcome: | The proposed approach is effective in assisting the LLMs adapt to multi-grained NLU tasks under zero-shot and few-shot multi-domain settings. |
Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) internalize a great amount of knowledge, but have been shown incapable of recalling this knowledge to solve complex & multi-step reasoning tasks. |
| Approach: | They propose an iterative prompting framework which progressively elicits relevant knowledge from PLMs for multi-step inference. |
| Outcome: | The proposed prompting framework outperforms existing prompting methods on three datasets involving multi-step reasoning. |
Re-Reading Improves Reasoning in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Unlike thought-eliciting prompting methods, RE2 shifts the focus to the input by processing questions twice, thereby enhancing the understanding process. |
| Approach: | They introduce a simple, yet general and effective prompting method, RE2, which rereads the question as input. |
| Outcome: | The proposed method demonstrates strong generality and compatibility with most thought-eliciting prompting methods, including CoT. |
What Makes Chain-of-Thought Prompting Effective? A Counterfactual Study (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using a few-shot prompt, we examine the effects of symbols and patterns on in-context learning in large language models. |
| Approach: | They employ a counterfactual prompting approach by manipulating examples and testing the consequences on model behavior. |
| Outcome: | The proposed approach allows us to understand the relative contributions of symbols and patterns on in-context learning. |
Metacognitive Prompting Improves Understanding in Large Language Models (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent advances in prompting have enhanced reasoning in logic-intensive tasks for LLMs, yet the nuanced understanding abilities of these models remain underexplored. |
| Approach: | They propose a strategy inspired by human introspective reasoning processes to enhance LLMs' understanding abilities. |
| Outcome: | The proposed method outperforms chain-of-thought prompting and its advanced versions on ten natural language understanding (NLU) datasets. |
Exploring Chain of Thought Style Prompting for Text-to-SQL (2023.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning with large language models (LLMs) has recently caught increasing attention due to its superior few-shot performance on various tasks. |
| Approach: | They propose a new chain of thought prompting method that enhances LLMs’ reasoning ability through chain of thinking prompting, including the original chain-of-thought prompting and least-to-most prompting. |
| Outcome: | The proposed method brings 5.2 and 6.5 point absolute gains on the Spider development set and the Spider Realistic set, respectively, compared to the standard prompting method without reasoning steps; 2.4 and 1.5 point absolute gain, versus the least-to-most prompting. |