FinePrompt: Unveiling the Role of Finetuned Inductive Bias on Compositional Reasoning in GPT-4 (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models such as GPT-4 have demonstrated impressive capability to solve textual understanding problems at a level parallel to or surpassing state-of-the-art taskspecific models. |
| Approach: | They propose to transfer task-specific inductive biases from finetuned models to prompts to improve GPT-4's compositional reasoning capabilities. |
| Outcome: | The proposed prompt scheme shows competitive zero-shot and few-shot performances compared to existing prompts on complicated reasoning tasks. |
Similar Papers
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023.acl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). |
| Approach: | They propose to use Chain-of-Thought (CoT) prompting to encourage the LLM to generate intermediate rationales for solving a problem by providing a series of reasoning steps in the demonstrations. |
| Outcome: | The proposed model can generate coherent lines of reasoning even with invalid demonstrations while still generating coherent lines during inference. |
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is popular in reasoning tasks, but its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. |
| Approach: | They propose a Coarse-to-Fine Chain-of-Thought approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire essential concepts. |
| Outcome: | The proposed approach is effective in assisting the LLMs adapt to multi-grained NLU tasks under zero-shot and few-shot multi-domain settings. |
Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting has been shown to enhance the multi-step reasoning capabilities of Large Language Models (LLMs). |
| Approach: | They propose to use CoT prompting to analyze a symbolic reasoning task where letters are shifted forward some number of steps in the alphabet. |
| Outcome: | The proposed model performs well on a symbolic reasoning task, with three LLMs performing the task using CoT prompts. |
Unveiling Confirmation Bias in Chain-of-Thought Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting has been widely adopted to enhance the reasoning capabilities of large language models (LLMs). |
| Approach: | They propose to examine how internal beliefs affect reasoning generation and reasoning-guided answer prediction in CoT by decomposing CoT into a two-stage process. |
| Outcome: | The proposed model beliefs affect reasoning generation and reasoning-guided answer prediction in CoT, and the results provide strong evidence of confirmation bias in LLMs. |
What Makes Chain-of-Thought Prompting Effective? A Counterfactual Study (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using a few-shot prompt, we examine the effects of symbols and patterns on in-context learning in large language models. |
| Approach: | They employ a counterfactual prompting approach by manipulating examples and testing the consequences on model behavior. |
| Outcome: | The proposed approach allows us to understand the relative contributions of symbols and patterns on in-context learning. |
Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting is a technique to enhance the reasoning abilities of Large language models (LLMs) however, the reasoning chains of demonstrations are observed to be prone to errors, which can lead to incorrect reasoning during inference. |
| Approach: | They propose an iterative bootstrapping technique to enhance the reasoning abilities of Large language models (LLMs) by generating a series of reasoning steps to obtain the answer, and using the reasoning chains as exemplars to demonstrate the task. |
| Outcome: | The proposed method improves the performance of Large language models (LLMs) on three reasoning tasks on ten datasets. |
Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a number of shortcomings, including lack of factual correctness. |
| Approach: | They propose a framework to increase prediction factuality by post-editing reasoning chains . they propose to use large language models to generate interpretable reasoning chains. |
| Outcome: | The proposed framework leads to accuracy improvements in open-domain question-answering tasks. |
Rethinking Prompt-based Debiasing in Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompt-based methods for debiasing are often superficial and lack a thorough understanding of complex bias concepts. |
| Approach: | They analyze a BBQ and stereoSet benchmarks to examine the assumption that large language models understand biases. |
| Outcome: | The proposed model misclassified 90% of unbiased content as biased despite high accuracy on BBQ dataset . the proposed model may have been flawed in previous attempts to debiase . |
R3 Prompting: Review, Rephrase and Resolve for Chain-of-Thought Reasoning in Large Language Models under Noisy Context (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have evaluated LLMs under noise-free context but the dilemma for LLM to produce inaccurate results under noisy context has not been fully investigated. |
| Approach: | They propose a new method for CoT reasoning using Chain-of-Thought prompting that interacts with LLMs to perform key sentence extraction, variable declaration and answer prediction. |
| Outcome: | The proposed method outperforms existing CoT prompting methods on five reasoning tasks under noisy context. |
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CoT synthesis approaches focus on simpler reasoning tasks and result in inconsistent CoT prompts. |
| Approach: | They propose a framework for automatic generation of superior CoT prompts based on three major evolution strategies . they propose 'step-level debating' method where multiple debaters discuss each reasoning step to arrive at the correct answer. |
| Outcome: | The proposed framework can generate superior CoT prompts from a CoT dataset. |