| Challenge: | Large Language Models (LLMs) have demonstrated impressive capabilities in reasoning, but how they propagate within their reasoning process remains underexplored. |
| Approach: | They propose a practical approach to mitigating misinformation propagation in LLMs by applying factual corrections early in the reasoning process and fine-tuning on synthesized data with early-stage corrections significantly improves reasoning factuality. |
| Outcome: | The proposed model can correct misinformation when explicitly instructed, but fails to correct misinformation less than half the time even with explicit instructions. |
Similar Papers
No Need for Explanations: LLMs can implicitly learn from mistakes in-context (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature assumes that correct answers to large language models must be accompanied by comprehensive rationales to be helpful. |
| Approach: | They propose to show incorrect answers to Large Language Models (LLMs) as a popular strategy to improve their performance in reasoning-intensive tasks. |
| Outcome: | The proposed approach outperforms chain-of-thought prompting in math reasoning tasks. |
Factuality of Large Language Models: A Survey (2024.emnlp-main)
Copied to clipboard
Yuxia Wang, Minghan Wang, Muhammad Arslan Manzoor, Fei Liu, Georgi Georgiev, Rocktim Das, Preslav Nakov
| Challenge: | Large language models (LLMs) are factually incorrect, which limits their applicability in real-world scenarios. |
| Approach: | They analyze existing work to identify major challenges and their associated causes . they propose to evaluate LLMs using a variety of measures to mitigate factual errors . |
| Outcome: | The proposed methods are based on a variety of datasets and proposed strategies to mitigate factual errors. |
Tracing and Dissecting How LLMs Recall Factual Knowledge for Real World Questions (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have shown promising ability to perform commonsense reasoning. |
| Approach: | They propose a two-dimensional analysis framework that incorporates token back-tracing and token decoding to uncover how LLMs conduct factual knowledge recall. |
| Outcome: | The proposed framework shows that LLMs lack relevant knowledge but struggle to select the most accurate information based on context during the retrieval and rerank phase. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
From Misleading Queries to Accurate Answers: A Three-Stage Fine-Tuning Method for LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods focus on correcting the output but overlook the ability of LLMs to detect and correct misleading content in the input itself. |
| Approach: | They propose a three-stage fine-tuning method that improves LLMs' ability to detect and correct misleading information in input queries. |
| Outcome: | The proposed method improves accuracy and factuality of LLM responses while also reducing hallucinations. |
Learn and Unlearn: Addressing Misinformation in Multilingual LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to unlearning large language models (LLMs) focus on English data, but they ignore multilingual contexts and can produce misleading, offensive, or otherwise fake content. |
| Approach: | They investigate the propagation of information in multilingual large language models and evaluate unlearning methods to address harmful content in multi-lingual contexts. |
| Outcome: | The proposed methods can effectively eliminate harmful content for all languages by addressing both English and the original language of the harmful data. |
Tracking the Limits of Knowledge Propagation: How LLMs Fail at Multi-Step Reasoning with Conflicting Knowledge (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for analyzing the performance of Large Language Models (LLMs) focus on single knowledge updates and fact recall, but do not consider how these updates affect downstream reasoning. |
| Approach: | They propose a benchmark to study how LLMs propagate new knowledge when it conflicts with the model's parametric knowledge. |
| Outcome: | The proposed benchmark compared models with no updated facts to show that the new methods worsen performance and improve reasoning performance. |
There’s No Such Thing as Simple Reasoning for LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing work has focused on relatively complex “many-hop” reasoning problems. |
| Approach: | They analyse the performance of fine-tuned LLMs on simple reasoning problems . they find the models remain highly brittle, being susceptible to seemingly innocent perturbations . |
| Outcome: | The proposed models fail on simple reasoning problems, but are highly brittle . they are susceptible to seemingly innocent perturbations, such as adding duplicates to the set of premises and shuffling the order in which the premises are presented. |
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated that a large language model (LLM) can generate training data for another LLM, or for creating supplementary training materials, such as rationales. |
| Approach: | They conduct an in-depth investigation to understand why fine-tuning an LLM with responses generated by a LLM often yields better results than using responses generated from humans. |
| Outcome: | The proposed approach can be used to transfer knowledge from a larger model to a smaller one, or for creating supplementary training materials, such as rationales. |
Exposing the Achilles’ Heel: Evaluating LLMs Ability to Handle Mistakes in Mathematical Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing evaluations focus on final accuracy, neglecting the critical aspect of reasoning capabilities. |
| Approach: | They propose to evaluate LLMs’ abilities to detect and correct reasoning mistakes by using rule-based methods and smaller language models. |
| Outcome: | The proposed model outperforms existing models such as GPT-4o and GPT4 in both accuracy and accuracy, but lacks data contamination and memorization concerns. |