Shuofei Qiao, Yixin Ou, Ningyu Zhang, Xiang Chen, Yunzhi Yao, Shumin Deng, Chuanqi Tan, Fei Huang, Huajun Chen
| Challenge: | Reasoning is an essential ability for complex problem-solving and can provide back-end support for various real-world applications. |
| Approach: | They present cutting-edge research on reasoning with language model prompting and provide systematic resources to help beginners. |
| Outcome: | The proposed approaches have not been systematically reviewed and analyzed. |
Similar Papers
Towards Reasoning in Large Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | Reasoning is a fundamental aspect of human intelligence that plays a crucial role in many intellectual activities. |
| Approach: | They propose to improve LLMs' ability to elicit reasoning by providing exemplars or prompts to model reasoning. |
| Outcome: | This paper provides a comprehensive overview of the state of knowledge on reasoning in large language models. |
Teaching-Inspired Integrated Prompting Framework: A Novel Approach for Enhancing Reasoning in Large Language Models (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit impressive performance across various domains but struggle with arithmetic reasoning tasks. |
| Approach: | They propose a Teaching-Inspired Integrated Prompting Framework which emulates the instructional process of a teacher guiding students. |
| Outcome: | The proposed framework improves reasoning accuracy on nine benchmarks. |
How Can We Know What Language Models Know? (2020.tacl-1)
Copied to clipboard
| Challenge: | Recent work examines knowledge contained in language models by having the LM fill in the blanks of prompts such as “Obama is a __ by profession”. |
| Approach: | They propose mining-based and paraphrasing-based methods to automatically generate high-quality and diverse prompts, as well as ensemble methods to combine answers from different prompts. |
| Outcome: | The proposed methods improve accuracy from 31.1% to 39.6% on the LAMA benchmark for extracting relational knowledge from LMs. |
Generated Knowledge Prompting for Commonsense Reasoning (2022.acl-long)
Copied to clipboard
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Peter West, Ronan Le Bras, Yejin Choi, Hannaneh Hajishirzi
| Challenge: | Existing methods for commonsense reasoning rely on high-quality knowledge, but they are often dominated by large-scale pretrained models that are fine-tuned on a target benchmark. |
| Approach: | They develop generated knowledge prompting which generates knowledge from a language model and provides it as additional input when answering a question. |
| Outcome: | The proposed method improves state-of-the-art models on four commonsense reasoning tasks. |
A Survey of Multilingual Reasoning in Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | This survey provides the first in-depth review of multilingual reasoning in Language Models. |
| Approach: | This survey provides the first in-depth review of multilingual reasoning in LMs. |
| Outcome: | The present study provides the first in-depth review of multilingual reasoning in LMs. |
What Makes a Good Natural Language Prompt? (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies on prompt quality show imbalanced support across models and tasks, and research gaps. |
| Approach: | They propose a property- and human-centric framework for evaluating prompt quality . they propose comparing prompt quality to other factors such as adverbs and apverbs . |
| Outcome: | The proposed framework reveals imbalanced support across models and tasks and substantial research gaps. |
Iteratively Prompt Pre-trained Language Models for Chain of Thought (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) internalize a great amount of knowledge, but have been shown incapable of recalling this knowledge to solve complex & multi-step reasoning tasks. |
| Approach: | They propose an iterative prompting framework which progressively elicits relevant knowledge from PLMs for multi-step inference. |
| Outcome: | The proposed prompting framework outperforms existing prompting methods on three datasets involving multi-step reasoning. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
How Interpretable are Reasoning Explanations from Prompting Large Language Models? (2024.findings-naacl)
Copied to clipboard
| Challenge: | Prompt Engineering has garnered significant attention for enhancing the performance of large language models across a multitude of tasks. |
| Approach: | They propose a simple prompting technique that yields more than 70% improvement in interpretability. |
| Outcome: | The proposed method improves interpretability by 70% across multiple dimensions. |
Frugal Prompting for Dialog Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are used in natural language processing tasks with an unrealistic speed and effectiveness. |
| Approach: | They propose more compact ways of providing dialog history information while ensuring good performance and reducing model’s inference-API costs. |
| Outcome: | The proposed models have the optimal usable-information density while maintaining good performance and reducing model’s inference-API costs. |