| Challenge: | Conceptual abstraction and analogy-making are crucial for human learning, reasoning, and adapting to unfamiliar domains. |
| Approach: | They propose a multi-stage progressive reasoning prompt framework A3E which is based on the structure mapping theory from cognitive psychology and efficiently annotates candidate story pairs across six fine-grained categories. |
| Outcome: | The proposed framework achieves an average performance gain of + 73% across a range of prompting baselines and base LLMs. |
Similar Papers
StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding (2023.emnlp-main)
Copied to clipboard
Cheng Jiayang, Lin Qiu, Tsz Chan, Tianqing Fang, Weiqi Wang, Chunkit Chan, Dongyu Ru, Qipeng Guo, Hongming Zhang, Yangqiu Song, Yue Zhang, Zheng Zhang
| Challenge: | Analogy-making between narratives is crucial for human reasoning . despite its importance, there has been limited research on story analogies . |
| Approach: | They construct a large-scale story-level analogy corpus with 24K story pairs . they find that the tasks are incredibly difficult for large language models such as ChatGPT . |
| Outcome: | The proposed corpus contains 24K story pairs from diverse domains with human annotations on two similarities from the extended Structure-Mapping Theory. |
ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base (2024.acl-long)
Copied to clipboard
| Challenge: | ANALOGYKB is a million-scale analogy knowledge base based on existing knowledge graphs (KGs) based upon relational knowledge triples, we can discover new analogies using the corresponding relations between concepts. |
| Approach: | They propose a million-scale analogy knowledge base derived from existing knowledge graphs (KGs) ANALOGYKB identifies analogies of the same relations and analogies from analogous relations . |
| Outcome: | The proposed model enables both smaller LMs and LLMs to gain better analogical reasoning capabilities. |
On the Relationship between Sentence Analogy Identification and Sentence Structure Encoding in Large Language Models (2024.findings-eacl)
Copied to clipboard
Thilini Wijesiriwardene, Ruwan Wickramarachchi, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha, Amit Sheth, Amitava Das
| Challenge: | Analogies facilitate the transfer of meaning and knowledge from one domain to another. |
| Approach: | They propose to use large language models to encode syntactic and semantic structures of sentences to identify sentence analogies. |
| Outcome: | The LLMs which capture syntactic structures better, also have higher abilities in identifying sentence analogies. |
In-Context Analogical Reasoning with Pre-Trained Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Analogical reasoning is a fundamental capacity of human cognition that allows us to reason abstractly about novel situations by relating them to past experiences. |
| Approach: | They apply large pre-trained language models to visual Raven’s Progressive Matrices (RPM) and use language-based abstractions to support analogy in AI systems. |
| Outcome: | The proposed language-based abstractions outperform human models on Raven’s Progressive Matrices and supervised vision-based methods. |
Inductive Linguistic Reasoning with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Evaluating large language models (LLMs) on their linguistic reasoning capabilities is an important task to understand the gaps in their skills that may surface during large-scale adoption. |
| Approach: | They propose to generate analogical exemplars with a language model and apply them in-context with target language exemplar. |
| Outcome: | The proposed method can be applied to other tasks present in Linguistics Olympiad competitions and achieves state-of-the-art results across nearly all problem types and difficulty levels in the LINGOLY dataset. |
KnowledgePrompts: Exploring the Abilities of Large Language Models to Solve Proportional Analogies via Knowledge-Enhanced Prompting (2025.coling-main)
Copied to clipboard
Thilini Wijesiriwardene, Ruwan Wickramarachchi, Sreeram Reddy Vennam, Vinija Jain, Aman Chadha, Amitava Das, Ponnurangam Kumaraguru, Amit Sheth
| Challenge: | Proportional analogies are used to assess linguistic and cognitive abilities. |
| Approach: | They propose a dataset for proportional analogy completion and evaluate its performance in large-scale learning environments. |
| Outcome: | The proposed model achieves 55% accuracy in knowledge-enhanced prompts. |
ANALOGICAL - A Novel Benchmark for Long Text Analogy Evaluation in Large Language Models (2023.findings-acl)
Copied to clipboard
Thilini Wijesiriwardene, Ruwan Wickramarachchi, Bimal Gajera, Shreeyash Gowaikar, Chandan Gupta, Aman Chadha, Aishwarya Naresh Reganti, Amit Sheth, Amitava Das
| Challenge: | Modern large language models are evaluated on extrinsic measures based on benchmarks such as GLUE and SuperGLUE. |
| Approach: | They propose a benchmark to intrinsically evaluate large language models across a taxonomy of analogies of long text with six levels of complexity. |
| Outcome: | The proposed benchmark evaluates LLMs across a taxonomy of analogies of long text with six levels of complexity. |
Relevant or Random: Can LLMs Truly Perform Analogical Reasoning? (2025.findings-acl)
Copied to clipboard
| Challenge: | Analogical reasoning is a unique ability of humans to address unfamiliar challenges by transferring strategies from relevant past experiences. |
| Approach: | They propose to use self-generated random examples to improve performance on a variety of reasoning tasks by incorporating relevant examples from relevant past experiences. |
| Outcome: | The proposed methods achieve comparable or even better performance on GSM8K with random biological examples. |
AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies (2024.emnlp-main)
Copied to clipboard
Xiao Ye, Andrew Wang, Jacob Choi, Yining Lu, Shreya Sharma, Lingfeng Shen, Vijay Murari Tiyyala, Nicholas Andrews, Daniel Khashabi
| Challenge: | Analogical reasoning is an important part of human communication, says a new study . a benchmark to determine analogical reasoning ability in language models is needed . |
| Approach: | They propose to benchmark analogical reasoning ability in language models by collecting 340 analogies from human writings. |
| Outcome: | The proposed benchmark aims to determine analogical reasoning ability in language models. |
Scientific and Creative Analogies in Pretrained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing analogy datasets focus on a limited set of analogical relations with a high similarity of the two domains between which the analogy holds. |
| Approach: | They propose a dataset that encodes analogy in pretrained language models . they use a system that maps attributes and relational structures across dissimilar domains . |
| Outcome: | The proposed dataset shows that state-of-the-art models achieve low performance on analogy tasks . |