Assessing the Reasoning Capabilities of LLMs in the context of Evidence-based Claim Verification (2025.findings-acl)
Copied to clipboard
John Dougrez-Lewis, Mahmud Elahi Akhter, Federico Ruggeri, Sebastian Löbbers, Yulan He, Maria Liakata
| Challenge: | Large Language Models (LLMs) have shown remarkable proficiency in complex tasks where reasoning capabilities are paramount. |
| Approach: | They propose a framework to break down claims into atomic reasoning types needed for verification. |
| Outcome: | The proposed framework breaks down claims into atomic reasoning types needed for verification. |
Similar Papers
Claim Verification in the Age of Large Language Models: A Survey (2026.acl-srw)
Copied to clipboard
| Challenge: | Recent election cycles have seen a large number of false information spread across social media and news platforms. |
| Approach: | They propose a framework for automated claim verification using Large Language Models and Retrieval Augmented Generation. |
| Outcome: | The proposed frameworks are based on large-scale models and new methods such as Retrieval Augmented Generation (RAG). |
A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoning (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing models of large language models struggle with complex logical reasoning problems. |
| Approach: | They propose to use large language models to identify their own errors to improve their models' performance. |
| Outcome: | The proposed models can identify logical fallacies accurately and improve by themselves. |
Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs (2025.findings-naacl)
Copied to clipboard
| Challenge: | a new framework for complex reasoning with LLMs is developed to improve reasoning proof accuracy and interpretability. |
| Approach: | They propose to use LLMs to generate search logs that can be interpreted into human-readable reasoning proofs. |
| Outcome: | The proposed framework improves reasoning accuracy but lacks interpretability due to black-box nature of the solvers. |
Towards Reasoning in Large Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | Reasoning is a fundamental aspect of human intelligence that plays a crucial role in many intellectual activities. |
| Approach: | They propose to improve LLMs' ability to elicit reasoning by providing exemplars or prompts to model reasoning. |
| Outcome: | This paper provides a comprehensive overview of the state of knowledge on reasoning in large language models. |
Towards Effective Extraction and Evaluation of Factual Claims (2025.acl-long)
Copied to clipboard
| Challenge: | Lack of a standardized evaluation framework impedes assessment and comparison of claim extraction methods. |
| Approach: | They propose a framework for evaluating claim extraction in the context of fact-checking . they also introduce Claimify, an LLM-based claim extraction method . |
| Outcome: | The proposed evaluation framework outperforms existing methods in the evaluation of claim extraction methods. |
A Comprehensive Evaluation of Inductive Reasoning Capabilities and Problem Solving in Large Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Inductive reasoning is fundamental to both human and artificial intelligence. |
| Approach: | They evaluated the inductive reasoning abilities of current Large Language Models (LLMs) and their performance on symbolic tasks. |
| Outcome: | The proposed models fail on symbolic tasks and show that chain-of-thought prompts help them by decomposing the problem-solving process, but the LLMs learn limitedly. |
Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing claims verification models rely on annotated data, which is expensive to create at a large scale. |
| Approach: | They propose a model that can verify complex claims without annotated data . they leverage the in-context learning ability of Large Language Models to translate a claim into a First-Order-Logic clause . |
| Outcome: | The proposed model outperforms baseline models on three datasets . it performs well on the datasets, and the results are published online. |
Stress-Testing the Reasoning Competence of Language Models With Formal Proofs (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new battery of challenging but tractable logical inference tasks is developed on ProofGrid . the tasks include proof writing and proof checking across propositional and equational logic . |
| Approach: | They propose a new battery of challenging but tractable logical inference tasks on ProofGrid . they introduce two new tasks: proof inpainting and proof gap-filling . |
| Outcome: | The proposed model performs well on top-tier models but also shows systematic failure modes. |
CriticBench: Benchmarking LLMs for Critique-Correct Reasoning (2024.findings-acl)
Copied to clipboard
| Challenge: | CriticBench is a benchmark designed to assess LLMs’ abilities to critique and refine their reasoning across a variety of tasks. |
| Approach: | They propose a benchmark to assess LLMs' ability to critique and correct reasoning across a variety of tasks. |
| Outcome: | The proposed benchmark examines the performance of 17 large language models in generation, critique, and correction reasoning. |
From Informal to Formal – Incorporating and Evaluating LLMs on Natural Language Requirements to Verifiable Formal Proofs (2025.acl-long)
Copied to clipboard
Jialun Cao, Yaojie Lu, Meiziniu Li, Haoyang Ma, Haokun Li, Mengda He, Cheng Wen, Le Sun, Hongyu Zhang, Shengchao Qin, Shing-Chi Cheung, Cong Tian
| Challenge: | Recent studies in formal mathematical reasoning have shown an unstoppable growth trend. |
| Approach: | They constructed 18k high-quality instruction-response pairs across five mainstream formal specification languages and evaluated them against ten open-sourced LLMs. |
| Outcome: | The proposed model compared instruction-response pairs across five formal specification languages and found that the LLMs were good at writing proof segments when given either the code, or the detailed description of proof steps. |