Instructing Large Language Models to Identify and Ignore Irrelevant Conditions (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing CoT prompting methods elicited multi-step reasoning abilities of large language models (LLMs) but they were seriously confused by the irrelevant conditions, resulting in low accuracy. |
| Approach: | They propose a method that instructs large language models to identify and ignore irrelevant conditions and prompts them to verify the irrelevant conditions. |
| Outcome: | The proposed approach outperforms existing methods on MWPs with GPT-3.5-Turbo and I3C-Select. |
Similar Papers
CoF-CoT: Enhancing Large Language Models with Coarse-to-Fine Chain-of-Thought Prompting for Multi-domain NLU Tasks (2023.emnlp-main)
Copied to clipboard
| Challenge: | Chain-of-Thought prompting is popular in reasoning tasks, but its application to Large Language Models (LLMs) in Natural Language Understanding (NLU) is under-explored. |
| Approach: | They propose a Coarse-to-Fine Chain-of-Thought approach that breaks down NLU tasks into multiple reasoning steps where LLMs can learn to acquire essential concepts. |
| Outcome: | The proposed approach is effective in assisting the LLMs adapt to multi-grained NLU tasks under zero-shot and few-shot multi-domain settings. |
INFORM : Information eNtropy based multi-step reasoning FOR large language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated exceptional performance with dedicated Chain-of-Thought (CoT) prompts. |
| Approach: | They propose a new method by introducing information entropy as a criteria on for CoT prompt selection. |
| Outcome: | The proposed model outperforms existing models on seven reasoning benchmarks using two language models. |
Large Language Models are Better Reasoners with Self-Verification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to solve complex natural language processing tasks require multiple steps to verify the answers. |
| Approach: | They propose to use chain of thought prompting to solve reasoning tasks with large language models. |
| Outcome: | The proposed method can improve reasoning performance on arithmetic, commonsense, and logical reasoning datasets. |
Large Language Models as an Indirect Reasoner: Contrapositive and Contradiction for Automated Reasoning (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies have focused on improving the ability of Large Language Models to perform complex reasoning. |
| Approach: | They propose a Direct-Indirect Reasoning method that integrates DR and IR as parallel reasoning paths that are merged to derive the final answer. |
| Outcome: | The proposed method outperforms existing methods on four datasets related to logical reasoning and proof. |
R3 Prompting: Review, Rephrase and Resolve for Chain-of-Thought Reasoning in Large Language Models under Noisy Context (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have evaluated LLMs under noise-free context but the dilemma for LLM to produce inaccurate results under noisy context has not been fully investigated. |
| Approach: | They propose a new method for CoT reasoning using Chain-of-Thought prompting that interacts with LLMs to perform key sentence extraction, variable declaration and answer prediction. |
| Outcome: | The proposed method outperforms existing CoT prompting methods on five reasoning tasks under noisy context. |
It’s Not Easy Being Wrong: Large Language Models Struggle with Process of Elimination Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent research aims to unlock the reasoning capabilities of large language models (LLMs) chain-of-thought (COT) prompting can help LLMs reason toward correct answers, but its efficacy in reasoning toward incorrect answers is unexplored. |
| Approach: | They propose a task where large language models reason toward incorrect answers using chain-of-thought prompting. |
| Outcome: | The proposed task underperforms the strategy of choosing the correct answer on commonsense and scientific reasoning datasets. |
CAC-CoT: Connector-Aware Compact Chain-of-Thought for Efficient Reasoning Data Synthesis Across Dual-System Cognitive Tasks (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Long chain-of-thought (CoT) prompting often slows or even degrades performance on fast, intuitive "System-1" tasks. |
| Approach: | They introduce a method that deliberately restricts reasoning to a small, fixed set of connector phrases, steering the model toward concise and well-structured explanations. |
| Outcome: | The method achieves 85% on GSM8K and 40% on GPQA while also surpassing the baseline by over 20%. |
Can Small Language Models Help Large Language Models Reason Better?: LM-Guided Chain-of-Thought (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing frameworks for guiding a language model in reasoning tasks are limited by their tendency to generate low-quality rationales that are repetitive and vacuous. |
| Approach: | They propose a framework that leverages a lightweight language model for guiding a black-box large LM in reasoning tasks. |
| Outcome: | The proposed framework outperforms baselines in answer prediction accuracy. |
Automatic Model Selection with Large Language Models for Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Chain-of-Thought and Program-Aided Language Models offer different strengths and weaknesses. |
| Approach: | They propose a model selection method that uses a large language model to select between two different reasoning methods. |
| Outcome: | The proposed method shows significant performance improvements across eight reasoning datasets with Codex, ChatGPT, and GPT-4. |
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have recently been shown to deliver impressive performance in various NLP tasks. |
| Approach: | They propose a plan-and-solve (PS) prompting that includes a few manual steps to generate reasoning steps and improves the quality of generated reasoning steps. |
| Outcome: | The proposed strategy outperforms Zero-shot-CoT on ten reasoning problems and has comparable performance to 8-shot CoT prompting on the math reasoning problem. |