SalaMAnder: Shapley-based Mathematical Expression Attribution and Metric for Chain-of-Thought Reasoning (2025.findings-emnlp)
Copied to clipboard
Yue Xin, Chen Shen, Shaotian Yan, Xiaosong Yuan, Yaoming Wang, Xiaofeng Zhang, Chenxi Huang, Jieping Ye
| Challenge: | Chain-of-Thought prompting improves the math reasoning capability of large language models. |
| Approach: | They propose a method for attribution of component-level contributions in CoT reasoning using Shapley value and a stratified sampling algorithm that significantly reduces computational complexity. |
| Outcome: | The proposed method reduces computational complexity and provides robust correlations with model performance. |
Similar Papers
Enhancing Chain-of-Thoughts Prompting with Iterative Bootstrapping in Large Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Chain-of-thought (CoT) prompting is a technique to enhance the reasoning abilities of Large language models (LLMs) however, the reasoning chains of demonstrations are observed to be prone to errors, which can lead to incorrect reasoning during inference. |
| Approach: | They propose an iterative bootstrapping technique to enhance the reasoning abilities of Large language models (LLMs) by generating a series of reasoning steps to obtain the answer, and using the reasoning chains as exemplars to demonstrate the task. |
| Outcome: | The proposed method improves the performance of Large language models (LLMs) on three reasoning tasks on ten datasets. |
CoTAR: Chain-of-Thought Attribution Reasoning with Multi-level Granularity (2024.findings-emnlp)
Copied to clipboard
| Challenge: | State-of-the-art QA systems employ Large Language Models (LLMs) however, these models tend to hallucinate information in their responses. |
| Approach: | They propose an attribution-oriented Chain-of-Thought reasoning method to enhance attributions. |
| Outcome: | The proposed method outperforms existing models on context enhanced question-answering datasets and shows that it can be used to improve accuracy. |
ChainLM: Empowering Large Language Models with Improved Chain-of-Thought Prompting (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing CoT synthesis approaches focus on simpler reasoning tasks and result in inconsistent CoT prompts. |
| Approach: | They propose a framework for automatic generation of superior CoT prompts based on three major evolution strategies . they propose 'step-level debating' method where multiple debaters discuss each reasoning step to arrive at the correct answer. |
| Outcome: | The proposed framework can generate superior CoT prompts from a CoT dataset. |
Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive reasoning abilities when prompted with Chain-of-Thought (CoT). |
| Approach: | They propose to categorize Chain-of-X methods by taxonomies of nodes, i.e., the X in CoX, and application tasks, and then categorise them by taxanomies and discuss potential future directions. |
| Outcome: | The proposed methods are categorised by taxonomies of nodes, i.e., the X in CoX, and application tasks. |
Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters (2023.acl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting can dramatically improve the multi-step reasoning abilities of large language models (LLMs). |
| Approach: | They propose to use Chain-of-Thought (CoT) prompting to encourage the LLM to generate intermediate rationales for solving a problem by providing a series of reasoning steps in the demonstrations. |
| Outcome: | The proposed model can generate coherent lines of reasoning even with invalid demonstrations while still generating coherent lines during inference. |
MathPrompter: Mathematical Reasoning using Large Language Models (2023.acl-industry)
Copied to clipboard
| Challenge: | Recent advances in natural language processing (NLP) can be attributed to massive scaling of Large Language Models (LLMs). |
| Approach: | They propose a technique that improves performance of Large Language Models (LLMs) on arithmetic problems along with increased reliance in the predictions. |
| Outcome: | The proposed technique improves performance on arithmetic problems and increases confidence in the output results. |
Analyzing Chain-of-thought Prompting in Black-Box Large Language Models via Estimated V-information (2024.lrec-main)
Copied to clipboard
| Challenge: | Chain-of-Thought (CoT) prompting and large language models (LLMs) have shown great potential in improving performance on challenging reasoning tasks. |
| Approach: | They propose a new metric which extends the concept of pointwise V-information to black-box models and quantifies label-relevant new information introduced by CoT prompting. |
| Outcome: | The proposed metric extends the concept of pointwise V-information to black-box models, quantifying label-relevant new information introduced by CoT prompting beyond pre-existing label information. |
Prompting Large Language Models with Chain-of-Thought for Few-Shot Knowledge Base Question Generation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for question generation over knowledge bases rely on annotated data for fine-tuning . emergence of Large Language Models (LLMs) has shown impressive generalization ability in few-shot tasks. |
| Approach: | They propose to use a logical form to generate a question in a reasoning problem . they propose to extend the prompting method into a method that can generate questions in logical forms . |
| Outcome: | The proposed method outperforms baselines on three public KBQG datasets. |
CoTJudger: A Graph-Driven Framework for Automatic Evaluation of Chain-of-Thought Efficiency and Redundancy in LRMs (2026.findings-acl)
Copied to clipboard
Siyi Li, Jiajun Shi, Shiwen Ni, Ge Zhang, Shuaimin Li, Shijian Wang, Zhoufutu Wen, Yizhi LI, Hamid Alinejad-Rokny, Jiaheng Liu, Min Yang, Wenhao Huang
| Challenge: | Existing evaluations emphasize final accuracy or coarse token counts, and lack automated tools to separate essential logic from structural redundancy. |
| Approach: | They propose a graph-driven framework that quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path needed to reach a correct solution. |
| Outcome: | Evaluating 21 LRMs, the proposed framework quantifies reasoning efficiency by converting free-form CoTs into directed dependency graphs and extracting the Shortest Effective Path (SEP) needed to reach a correct solution. |
What Makes Chain-of-Thought Prompting Effective? A Counterfactual Study (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Using a few-shot prompt, we examine the effects of symbols and patterns on in-context learning in large language models. |
| Approach: | They employ a counterfactual prompting approach by manipulating examples and testing the consequences on model behavior. |
| Outcome: | The proposed approach allows us to understand the relative contributions of symbols and patterns on in-context learning. |