Chain of Logic: Rule-Based Reasoning with Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Logic models are prone to hallucinations and are not able to perform basic tasks like drafting and drafting documents. |
| Approach: | They propose a new prompting method which elicits rule-based reasoning through decomposition and recomposition. |
| Outcome: | The proposed method outperforms other prompting methods including chain of thought and self-ask on eight rule-based reasoning tasks. |
Similar Papers
LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models (2024.acl-long)
Copied to clipboard
Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra, Chitta Baral
| Challenge: | Existing work investigating the logical reasoning ability of large language models has focused only on a couple of inference rules of propositional and first-order logics. |
| Approach: | They propose to use a natural language question-answering dataset to evaluate the logical reasoning ability of large language models. |
| Outcome: | The proposed model performs poorly on a range of natural language questions using chain-of-thought prompting. |
Chain-of-Reasoning: Towards Unified Mathematical Reasoning in Large Language Models via a Multi-Paradigm Perspective (2025.acl-long)
Copied to clipboard
Yiyao Yu, Yuxiang Zhang, Dongdong Zhang, Xiao Liang, Hengyuan Zhang, Xingxing Zhang, Mahmoud Khademi, Hany Hassan Awadalla, Junjie Wang, Yujiu Yang, Furu Wei
| Challenge: | Existing work shows that LLMs rely on single-paradigm reasoning that limits their effectiveness across diverse tasks. |
| Approach: | They propose a new framework that integrates multiple reasoning paradigms to enable synergistic collaboration. |
| Outcome: | The proposed model outperforms current SOTA models in theorem proving tasks and the MATH benchmark in arithmetic tasks. |
Multi-Step Inference for Reasoning Over Paragraphs (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing models for complex reasoning use symbols or black-box transformers . a compositional model can chain together free-form predicates and logical connectives . |
| Approach: | They propose a compositional model that finds relevant sentences and then chains them together using neural modules. |
| Outcome: | The proposed model improves performance on a recently-introduced dataset. |
Can LLMs Reason with Rules? Logic Scaffolding for Stress-Testing and Improving LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive human-like performance across various reasoning tasks, but their mastery of underlying inferential rules falls short of human capabilities. |
| Approach: | They propose a logic scaffolding inferential rule generation framework to construct an infer- ential rule base, ULogic, comprising both primitive and compositional rules across five domains. |
| Outcome: | The proposed model improves the ability to generate accurate, complex and abstract conclusions and premises and improves various commonsense reasoning tasks. |
Bi-Chainer: Automated Large Language Models Reasoning with Bidirectional Chaining (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing unidirectional chaining methods suffer from low prediction accuracy and efficiency. |
| Approach: | They propose a bidirectional chaining method which dynamically switches to depth-first reasoning in the opposite reasoning direction when it encounters multiple branching options within the current direction. |
| Outcome: | The proposed method achieves sizable accuracy boots over unidirectional chaining frameworks on four challenging logical reasoning datasets. |
What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in reasoning with large language models have popularized Long Chain-of-Thought (LCoT) a framework that converts sequential LCoTs into hierarchical tree structures enables deeper structural analysis of LLM reasoning. |
| Approach: | They propose a framework that converts sequential LCoTs into hierarchical tree structures and enables deeper structural analysis of LLM reasoning. |
| Outcome: | The proposed framework can be used to analyze LLM reasoning in a variety of tasks and models. |
How Long Reasoning Chains Influence LLMs’ Judgment of Answer Factuality (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly adopted as scalable judges for open-ended generation, yet how they form judgments remains insufficiently understood. |
| Approach: | They show that exposing reasoning influences LLM-based judgment . they also show that reasoning fluency and factuality critically shape judgment outcomes . |
| Outcome: | Empirical results show that the presence of reasoning significantly alters judgment behavior . stronger judges exhibit more selective behavior and achieve higher judgment accuracy . |
Chain of Methodologies: Scaling Test Time Computation without Training (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing prompts for complex reasoning tasks are limited to specific tasks with few-shot examples due to constraints like context length and information extraction accuracy. |
| Approach: | They propose a method to build structured reasoning processes by injecting human insights into LLMs' training data. |
| Outcome: | The proposed framework outperforms baselines in the analysis of large language models. |
Logic-of-Thought: Injecting Logic into Contexts for Full Reasoning in Large Language Models (2025.naacl-long)
Copied to clipboard
Tongxuan Liu, Wenjiang Xu, Weizhe Huang, Yuting Zeng, Jiaxing Wang, Xingyu Wang, Hailong Yang, Jing Li
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks but their performance in complex logical reasoning tasks remains unsatisfactory. |
| Approach: | They propose a propositional logic prompting method which generates expanded logical information descriptions and utilizes them as an additional augmentation to original contexts. |
| Outcome: | Extensive experiments show that Logic-of-Thought boosts the performance of various prompting methods with a striking margin across five logical reasoning tasks. |
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains. |
| Approach: | They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple. |
| Outcome: | The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness. |