Self-attention-based Graph-of-Thought for Math Problem Solving (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for generating reasoning paths in a chain structure are inefficient and non-human-like. |
| Approach: | They propose a decoding method for a chain-based LLM that constructs a thought graph simultaneously as an LLM inference and generates reasoning steps with a graph-structured self-attention mechanism. |
| Outcome: | The proposed method improves reasoning accuracy without huge computational over-expensive LLMs and avoids performance degradation issues when the LLM is too small to comprehend complex prompts. |
Similar Papers
Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs (2024.findings-acl)
Copied to clipboard
Bowen Jin, Chulin Xie, Jiawei Zhang, Kashob Kumar Roy, Yu Zhang, Zheng Li, Ruirui Li, Xianfeng Tang, Suhang Wang, Yu Meng, Jiawei Han
| Challenge: | Existing studies suggest augmenting LLMs with external text corpora to alleviate hallucination problems. |
| Approach: | They propose to augment large language models with text units retrieved from external knowledge corpora to alleviate the issue. |
| Outcome: | The proposed framework outperforms baselines on GRBench with three LLMs and shows that iterative reasoning outperformed the baselines. |
GoT: Effective Graph-of-Thought Reasoning in Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have been advancing at an unprecedented pace. |
| Approach: | They propose a graph-based approach which models human thought processes as a chain and as 'graphs' by representing thought units as nodes and connections between them as edges, they capture the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. |
| Outcome: | The proposed model improves on a text-only reasoning task and a multimodal reasoning task. |
Mapping the Minds of LLMs: A Graph-Based Analysis of Reasoning LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models (LRMs) often display unstable behaviors, e.g., hallucinating unsupported premises, overthinking simple tasks, and displaying higher sensitivity to prompt variations. |
| Approach: | They propose a graph-based analytical framework that clusters long, verbose CoT outputs into semantically coherent reasoning steps, then constructs directed reasoning graphs to capture contextual and logical dependencies among these steps. |
| Outcome: | The proposed framework enables quantitative evaluation of internal reasoning structure and quality beyond conventional metrics and provides practical insights for prompt engineering and cognitive analysis of LLMs. |
DGoT: Dynamic Graph of Thoughts for Scientific Abstract Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for generating abstracts involve collecting domain data and training corresponding models to complete the task of text summarization. |
| Approach: | They propose a method to train language models based on domain datasets and a Dynamic Graph of Thought (DGoT) which inherits the advantages of existing GoT prompt approach while reducing model reasoning cost. |
| Outcome: | The proposed method saves the cost of model training and improves reliability due to the hallucination problem of LLMs. |
Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a new framework for self-attention models is proposed to address this problem . it injects *multi-hop* relationships into the attention graph, allowing for better performance . |
| Approach: | They propose a framework that injects *multi-hop* relationships through a belief propagation process. |
| Outcome: | The proposed framework helps prevent entropy collapse in deeper layers and maintains GTD at task-appropriate levels. |
Vision-Language Models Can Self-Improve Reasoning via Reflection (2025.naacl-long)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) has been shown to improve the reasoning capability of large language models (LLMs). |
| Approach: | They propose a framework which iteratively enhances the model’s Vision-language Reasoning by Reflecting on CoT Rationales. |
| Outcome: | The proposed framework improves multimodal reasoning on vision-language tasks by 23% to 60% over baselines. |
Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that inference-time scaling increases performance of Large Language Models (LLMs) in various reasoning tasks such as mathematics and complex question answering by increasing the length of Chain-of-Thought (CoT). |
| Approach: | They propose a model which synthesizes longer CoT data and iteratively improves performance through self-training by incorporating a few demonstration examples. |
| Outcome: | The proposed model achieves an average improvement of more than +2.5 points across five reasoning tasks: MMLU, GSM8K, ARC-C, HellaSwag, and BBH on two backbone models. |
Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have enabled strong reasoning capabilities through Chain-of-Thought (CoT) prompting. |
| Approach: | They propose a framework that integrates cognitively inspired reasoning paradigms with linguistic constraints to reduce token usage while preserving reasoning accuracy. |
| Outcome: | The proposed framework reduces token usage while preserving reasoning accuracy across 18 reasoning datasets across multiple domains, languages, and modalities. |
CoMAT: Chain of Mathematically Annotated Thought Improves Mathematical Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Mathematical reasoning remains a significant challenge for large language models (LLMs), despite advances in prompting techniques such as Chain-of-Thought (CoT). |
| Approach: | They propose a framework that enhances reasoning through two stages: Symbolic Conversion and Reasoning Execution. |
| Outcome: | The proposed framework outperforms traditional CoT on six out of seven benchmarks across four LLMs. |
ClozeMath: Improving Mathematical Reasoning in Language Models by Learning to Fill Equations (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods to train large language models do not capture how humans learn to think. |
| Approach: | They propose a method to fine-tune large language models for mathematical reasoning by using a text-infilling task that predicts masked equations from a given solution. |
| Outcome: | Experiments on GSM8K, MATH, and GSM-Symbolic show that ClozeMath surpasses baseline Masked Thought in performance and robustness with two test-time scaling decoding algorithms, Beam Search and Chain-of-Thought decoding. |