Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? (2025.findings-acl)
Copied to clipboard
| Challenge: | Analogical reasoning is a unique ability of humans to address unfamiliar challenges by transferring strategies from relevant past experiences. |
| Approach: | They propose to use self-generated random examples to improve performance on a variety of reasoning tasks by incorporating relevant examples from relevant past experiences. |
| Outcome: | The proposed methods achieve comparable or even better performance on GSM8K with random biological examples. |
Similar Papers
I Learn Better If You Speak My Language: Understanding the Superior Performance of Fine-Tuning Large Language Models with LLM-Generated Responses (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent research has demonstrated that a large language model (LLM) can generate training data for another LLM, or for creating supplementary training materials, such as rationales. |
| Approach: | They conduct an in-depth investigation to understand why fine-tuning an LLM with responses generated by a LLM often yields better results than using responses generated from humans. |
| Outcome: | The proposed approach can be used to transfer knowledge from a larger model to a smaller one, or for creating supplementary training materials, such as rationales. |
Can language models learn analogical reasoning? Investigating training objectives and comparisons to human performance (2023.emnlp-main)
Copied to clipboard
| Challenge: | Analogical reasoning is a common way to evaluate word embeddings in NLP, but it is also of interest to investigate whether or not it is able to be learned. |
| Approach: | They propose to use proportional analogies to evaluate word embeddings in NLP . they also test whether analogical reasoning is a task in itself that can be learned . |
| Outcome: | The proposed models can learn analogical reasoning even with small amounts of data. |
AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies (2024.emnlp-main)
Copied to clipboard
Xiao Ye, Andrew Wang, Jacob Choi, Yining Lu, Shreya Sharma, Lingfeng Shen, Vijay Murari Tiyyala, Nicholas Andrews, Daniel Khashabi
| Challenge: | Analogical reasoning is an important part of human communication, says a new study . a benchmark to determine analogical reasoning ability in language models is needed . |
| Approach: | They propose to benchmark analogical reasoning ability in language models by collecting 340 analogies from human writings. |
| Outcome: | The proposed benchmark aims to determine analogical reasoning ability in language models. |
Out-of-Context Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a lightweight technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
| Approach: | They propose a lightweight technique that trains only new token embeddings on axioms . they train only new embeddables and evaluate them on unseen tasks . |
| Outcome: | The proposed technique trains only new token embeddings on axioms and evaluates them on unseen tasks. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
Can LLMs Learn from Previous Mistakes? Investigating LLMs’ Errors to Boost for Reasoning (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown the benefits to LLMs from fine-tuning golden-standard Chain-of-Thought rationales or using them as correct examples in few-shot prompting. |
| Approach: | They propose a new benchmark to test the effectiveness of large language models by leveraging errors to enhance reasoning capabilities. |
| Outcome: | The proposed methods can be used to fine-tune models in correct and incorrect domains, rather than tuning models to learn ground truth in traditional methods. |
No Need for Explanations: LLMs can implicitly learn from mistakes in-context (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature assumes that correct answers to large language models must be accompanied by comprehensive rationales to be helpful. |
| Approach: | They propose to show incorrect answers to Large Language Models (LLMs) as a popular strategy to improve their performance in reasoning-intensive tasks. |
| Outcome: | The proposed approach outperforms chain-of-thought prompting in math reasoning tasks. |
Improving LLM Generations via Fine-Grained Self-Endorsement (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) have demonstrated remarkable capabilities but can still fail frequently on knowledge-intensive tasks. |
| Approach: | They propose a self-endorsement framework that leverages fine-grained fact-level comparisons across multiple sampled responses. |
| Outcome: | The proposed framework can improve factuality of generations with simple prompts across scales of LLMs. |
Modelling Analogies and Analogical Reasoning: Connecting Cognitive Science Theory and NLP Research (2026.tacl-1)
Copied to clipboard
| Challenge: | Analogical reasoning is an essential aspect of human cognition, says aaron eliotta . eelisa e. sabet: some have argued that analogy is central to the human cognitive experience . |
| Approach: | They summarize key theories about the processes underlying analogical reasoning from the cognitive science literature and relate it to current research in natural language processing. |
| Outcome: | The proposed approaches are relevant for several major challenges in natural language processing, not directly related to analogy solving. |
How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark (2025.emnlp-main)
Copied to clipboard
| Challenge: | Prior work has not explored the mechanisms underlying this sensitivity. |
| Approach: | They propose a synthetic benchmark to evaluate Large Language Models’ reasoning robustness against systematically controlled irrelevant context (IC). |
| Outcome: | The proposed model improves in-distribution and out-of-disttribution scenarios while training with strong distractors. |