SymBa: Symbolic Backward Chaining for Structured Natural Language Reasoning (2025.naacl-long)
Copied to clipboard
| Challenge: | Among different methods for structured reasoning, we focus on backward chaining, where the goal is recursively decomposed into subgoals by searching and applying rules. |
| Approach: | They propose a backward chaining system that integrates a symbolic solver and an LLM to improve the performance of LLM-based reasoning. |
| Outcome: | The proposed system improves deductive, relational, and arithmetic reasoning benchmarks compared to baselines. |
Similar Papers
LAMBADA: Backward Chaining for Automated Reasoning in Natural Language (2023.acl-long)
Copied to clipboard
| Challenge: | Recent advances in automated reasoning with natural text suffer from a combinatorial explosion of the search space and high failure rates for problems requiring longer chains of reasoning. |
| Approach: | They propose a Backward Chaining algorithm that decomposes reasoning into four sub-modules and implements it by few-shot prompted LLM inference. |
| Outcome: | The proposed algorithm achieves sizable accuracy boosts over state-of-the-art forward reasoning methods on two challenging logical reasoning datasets. |
Faithful Logical Reasoning via Symbolic Chain-of-Thought (2024.acl-long)
Copied to clipboard
| Challenge: | SymbCoT is a framework that integrates symbolic expressions and logic rules with CoT prompting. |
| Approach: | They propose a Symbolic Chain-of-Thought framework that integrates symbolic expressions and logic rules with CoT prompting. |
| Outcome: | The proposed framework improves on 5 standard datasets with symbolic expressions and rules . it shows that it is more faithful, flexible, and explainable than the current method . |
Evaluating Step-by-Step Reasoning through Symbolic Verification (2024.findings-naacl)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs) have shown remarkable reasoning performance using explanations or chain-of-thoughts (CoT)) for in-context learning. |
| Approach: | They propose to use symbolic examples to iteratively reason over symbolic examples and to recover Prolog’s backward chaining algorithm to iterate over KBs. |
| Outcome: | The proposed model performs better on length generalization benchmarks than CoT on explanations and chain-of-thoughts (CoT) tasks. |
BC-Prover: Backward Chaining Prover for Formal Theorem Proving (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for interactive theorem proving in formal logic lack robustness and robustness. |
| Approach: | They propose a backward chaining framework guided by pseudo steps for proofstep generation that prioritizes pseudo steps. |
| Outcome: | The proposed framework improves on the miniF2F benchmark. |
Adaptive LLM-Symbolic Reasoning via Dynamic Logical Solver Composition (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to NLP are static and require manual formalization. |
| Approach: | They propose an adaptive, multi-paradigm, neuro-symbolic inference framework that automatically identifies formal reasoning strategies from problems expressed in natural language and dynamically selects and applies specialized formal logical solvers. |
| Outcome: | The proposed framework outperforms baselines on individual and multi-paradigm reasoning tasks by 17% and 6%. |
Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs (2025.findings-naacl)
Copied to clipboard
| Challenge: | a new framework for complex reasoning with LLMs is developed to improve reasoning proof accuracy and interpretability. |
| Approach: | They propose to use LLMs to generate search logs that can be interpreted into human-readable reasoning proofs. |
| Outcome: | The proposed framework improves reasoning accuracy but lacks interpretability due to black-box nature of the solvers. |
LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers (2023.emnlp-main)
Copied to clipboard
Theo Olausson, Alex Gu, Ben Lipkin, Cedegao Zhang, Armando Solar-Lezama, Joshua Tenenbaum, Roger Levy
| Challenge: | Logical reasoning is an important task for artificial intelligence, says a new study . many prompting-based strategies to enable large language models fail in subtle and unpredictable ways. |
| Approach: | They propose to reformulate logical reasoning tasks by leveraging large language models . they use a modular neurosymbolic programming approach to translate premises and conclusions from natural language to logic . |
| Outcome: | The proposed approach outperforms open-source models on FOLIO and ProofWriter while showing distinct failure modes. |
From Informal to Formal – Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs (2025.acl-long)
Copied to clipboard
Jialun Cao, Yaojie Lu, Meiziniu Li, Haoyang Ma, Haokun Li, Mengda He, Cheng Wen, Le Sun, Hongyu Zhang, Shengchao Qin, Shing-Chi Cheung, Cong Tian
| Challenge: | Recent studies in formal mathematical reasoning have shown an unstoppable growth trend. |
| Approach: | They constructed 18k high-quality instruction-response pairs across five mainstream formal specification languages and evaluated them against ten open-sourced LLMs. |
| Outcome: | The proposed model compared instruction-response pairs across five formal specification languages and found that the LLMs were good at writing proof segments when given either the code, or the detailed description of proof steps. |
Meta-Reasoning: Semantics-Symbol Deconstruction for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing methods rely on syntactically mapping natural languages to complete formal languages like Python and SQL. |
| Approach: | They propose to deconstruct reasoning-independent semantic information into generic symbolic representations, thereby efficiently capturing more generalized reasoning knowledge. |
| Outcome: | The proposed method improves in-context reasoning accuracy, learning efficiency, out-of-domain generalization, and output stability compared to the Chain-of thought technique. |
LLMs Faithfully and Iteratively Compute Answers During CoT: A Systematic Analysis With Multi-step Arithmetics (2026.findings-eacl)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Shusaku Sone, Masaya Taniguchi, Ana Brassard, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Specifically, we examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer. |
| Approach: | They examine when the LLMs’ answer is (pre)determined, especially before the CoT begins or after, and how strongly the information from CoT specifically has a causal effect on the final answer. |
| Outcome: | The proposed model can generate reasoning chains while generating the reasoning chain on the fly. |