PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents (2024.eacl-long)
Copied to clipboard
| Challenge: | Using chain-of-thought prompting, large language models perform better on complex reasoning tasks. |
| Approach: | They propose a prompting framework that decomposes a question into a sequence of actions and executes them over the document to obtain the answer. |
| Outcome: | The proposed framework outperforms zero-shot and chain-of-thought prompting on a QuALITY dataset . it proposes a plan based on actions mined from a training set and executes it step by step . |
Similar Papers
Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have recently been shown to deliver impressive performance in various NLP tasks. |
| Approach: | They propose a plan-and-solve (PS) prompting that includes a few manual steps to generate reasoning steps and improves the quality of generated reasoning steps. |
| Outcome: | The proposed strategy outperforms Zero-shot-CoT on ten reasoning problems and has comparable performance to 8-shot CoT prompting on the math reasoning problem. |
Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for question generation over knowledge bases rely on annotated data for fine-tuning . emergence of Large Language Models (LLMs) has shown impressive generalization ability in few-shot tasks. |
| Approach: | They propose to use a logical form to generate a question in a reasoning problem . they propose to extend the prompting method into a method that can generate questions in logical forms . |
| Outcome: | The proposed method outperforms baselines on three public KBQG datasets. |
Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot Relation Extraction (RE) lack detailed, context-specific prompts for understanding various sentences and relations. |
| Approach: | They propose a framework that uses a three-stage diversity approach to prompt LLMs by generating multiple synthetic samples that encapsulate specific relations from scratch. |
| Outcome: | The proposed framework outperforms existing LLM-based zero-shot RE methods on benchmark datasets and shows that it produces high-quality synthetic data that enhances performance. |
Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) excel at straightforward reasoning tasks, but struggle when faced with complex multi-step reasoning. |
| Approach: | They propose a framework that converts unstructured text into a graph and instructs LLMs to navigate this graph using task-specific strategies. |
| Outcome: | The proposed framework improves the multi-step reasoning capabilities of Large Language Models in a zero-shot setting. |
Hierarchical Prompting Assists Large Language Model on Web Navigation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models struggle on processing complicated observations in interactive decision making tasks. |
| Approach: | They propose a hierarchical prompting approach that constructs an action-aware observation and a Summarizer prompt. |
| Outcome: | The proposed method outperforms the current state-of-the-art prompting mechanism by 6.2% on task success rate. |
Making Large Language Models Better Reasoners with Orchestrated Streaming Experiences (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies show that large language models can perform complex reasoning tasks without labeled data and unlabeled data. |
| Approach: | They propose a framework for solving reasoning tasks that store answers in a streaming experience pool and orchestrate helpful questions from the pool to assist itself in answering new questions. |
| Outcome: | The proposed framework can self-improve as it answers reasoning questions . it stores all answered reasoning questions and their reasoning steps in a streaming experience pool . |
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CoT synthesis approaches focus on simpler reasoning tasks and result in inconsistent CoT prompts. |
| Approach: | They propose a framework for automatic generation of superior CoT prompts based on three major evolution strategies . they propose 'step-level debating' method where multiple debaters discuss each reasoning step to arrive at the correct answer. |
| Outcome: | The proposed framework can generate superior CoT prompts from a CoT dataset. |
Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) internalize a great amount of knowledge, but have been shown incapable of recalling this knowledge to solve complex & multi-step reasoning tasks. |
| Approach: | They propose an iterative prompting framework which progressively elicits relevant knowledge from PLMs for multi-step inference. |
| Outcome: | The proposed prompting framework outperforms existing prompting methods on three datasets involving multi-step reasoning. |
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains. |
| Approach: | They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple. |
| Outcome: | The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness. |
Let’s Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts to improve CoT prompting have limitations that require extensive human effort or performance needs to be improved. |
| Approach: | They propose a prompt approach for automatic reasoning called LBS3 inspired by curriculum learning which better reflects human learning habits. |
| Outcome: | The proposed approach achieves strongly competitive performance compared to baselines in reasoning-intensive tasks with varying open- and closed-source LLMs. |