ThinkSum: Probabilistic reasoning over sets using large language models (2023.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a substantial capacity for high-level analogical reasoning, but they fail in scenarios that require reasoning over multiple objects or facts and making sequences of logical deductions. |
| Approach: | They propose a two-stage probabilistic inference paradigm, ThinkSum, which reasons over sets of objects or facts in a structured manner. |
| Outcome: | The proposed paradigm improves on the BIG-bench suite of evaluation tasks. |
Similar Papers
Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models (2024.emnlp-main)
Copied to clipboard
Hyungjoo Chae, Yeonghyeon Kim, Seungone Kim, Kai Ong, Beong-woo Kwak, Moohyeon Kim, Sunghwan Kim, Taeyoon Kwon, Jiwan Chung, Youngjae Yu, Jinyoung Yeo
| Challenge: | Prior work has used LLMs to generate programming language and applied external compilers for such tasks. |
| Approach: | They propose a framework that expresses task-level logic with pseudocode and tailors it to each instance and simulates execution of it. |
| Outcome: | The proposed framework outperforms baselines in diverse reasoning tasks. |
Drift: Enhancing LLM Faithfulness in Rationale Generation via Dual-Reward Probabilistic Inference (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to improving LLM faithfulness rely on superficial calibration methods or costly retraining. |
| Approach: | They propose a probabilistic inference paradigm that leverages task-specific and lookahead rewards to ensure that LLM-generated rationales are more faithful to model decisions. |
| Outcome: | The proposed model improves both accuracy and faithfulness of Large Language Models (LLMs) on three reasoning tasks. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
Let’s Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts to improve CoT prompting have limitations that require extensive human effort or performance needs to be improved. |
| Approach: | They propose a prompt approach for automatic reasoning called LBS3 inspired by curriculum learning which better reflects human learning habits. |
| Outcome: | The proposed approach achieves strongly competitive performance compared to baselines in reasoning-intensive tasks with varying open- and closed-source LLMs. |
Improving Consistency in LLM Inference using Probabilistic Tokenization (2025.findings-naacl)
Copied to clipboard
| Challenge: | Prior work has shown that probabilistic tokenizations can generate multiple tokenization of the same input string. |
| Approach: | They propose a method to leverage the multiple tokenization capabilities of modern LLM tokenizers. |
| Outcome: | The proposed method improves the self-consistency of large language models by generating multiple tokenizations. |
Large Language Models are few(1)-shot Table Reasoners (2023.findings-eacl)
Copied to clipboard
| Challenge: | Recent literature has shown that large language models are excellent few-shot reasoners to solve text reasoning tasks. |
| Approach: | They evaluated LLMs on popular table QA and fact verification datasets like WikiTableQuestion, FetaQA, TabFact, and FEVEROUS and found they are competent at complex reasoning over table structures. |
| Outcome: | The proposed models are more competent at complex reasoning over table structures than tuned T5-large models. |
Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have enabled strong reasoning capabilities through Chain-of-Thought (CoT) prompting. |
| Approach: | They propose a framework that integrates cognitively inspired reasoning paradigms with linguistic constraints to reduce token usage while preserving reasoning accuracy. |
| Outcome: | The proposed framework reduces token usage while preserving reasoning accuracy across 18 reasoning datasets across multiple domains, languages, and modalities. |
LLM2: Let Large Language Models Harness System 2 Reasoning (2025.naacl-short)
Copied to clipboard
| Challenge: | Empirical results on mathematical reasoning benchmarks substantiate the efficacy of Large language models (LLMs). |
| Approach: | They propose a framework that combines an LLM with a process-based verifier to generate plausible candidates and provide timely process-driven feedback to distinguish desirable and undesirable outputs. |
| Outcome: | Empirical results show that LLM2 improves accuracy on GSM8K and self-consistency increases major@20 accuracy. |
LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models (2024.acl-long)
Copied to clipboard
Mihir Parmar, Nisarg Patel, Neeraj Varshney, Mutsumi Nakamura, Man Luo, Santosh Mashetty, Arindam Mitra, Chitta Baral
| Challenge: | Existing work investigating the logical reasoning ability of large language models has focused only on a couple of inference rules of propositional and first-order logics. |
| Approach: | They propose to use a natural language question-answering dataset to evaluate the logical reasoning ability of large language models. |
| Outcome: | The proposed model performs poorly on a range of natural language questions using chain-of-thought prompting. |