Does Self-Consistency Improve the Recall of Encyclopedic Knowledge? (2026.acl-short)
Copied to clipboard
| Challenge: | a lack of evaluation grounds for self-consistency on symbolic reasoning is unclear . however, it is unclear whether it improves performance on non-math questions involving encyclopedic knowledge. |
| Approach: | They establish a knowledge recall split for the popular MMLU benchmark by applying a data-driven heuristic from prior work. |
| Outcome: | The proposed knowledge recall split achieves an 89% accuracy on the MMLU benchmark. |
Similar Papers
Confidence Improves Self-Consistency in LLMs (2025.findings-acl)
Copied to clipboard
| Challenge: | Modern large language models (LLMs) demonstrate strong reasoning capabilities, driven in part by their capacity to generate a sequence of intermediate reasoning steps that lead them toward a final answer. |
| Approach: | They propose a method that performs a weighted majority vote based on confidence scores obtained directly from the model. |
| Outcome: | The proposed method outperforms self-consistency on nine models and four datasets, reducing the required number of reasoning paths by over 40% on average. |
Self-Para-Consistency: Improving Reasoning Tasks at Low Cost for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that self-consistency decoding can improve performance for complex reasoning tasks with large language models. |
| Approach: | They propose a self-consistency decoding strategy that generates multiple paraphrases for each test question and then generates reasoning paths for the original and all the paraphrased questions based on greedy decoding. |
| Outcome: | The proposed strategy reduces the sampling number and improves performance on complex reasoning tasks. |
Self-Consistency Boosts Calibration for Math Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing solutions for math reasoning tasks use semantic parsing or AST decoding, but performance can degrade dramatically even with slight changes to the questions. |
| Approach: | They propose three calibration methods based on self-consistency for math reasoning tasks. |
| Outcome: | The proposed methods bridge model confidence and accuracy better than existing methods based on p(True) or logit. |
Self-Consistency from Only Two Samples: CoT–PoT Ensembling for Efficient LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Self-consistency (SC) is a popular technique for improving the reasoning accuracy of large language models but it comes at a high computational cost due to extensive sampling. |
| Approach: | They propose a hybrid ensembling approach that leverages the complementary strengths of Chain-of-Thought and Program-of -Thus . they propose encapsulating two different modes of reasoning to create a single output and a final answer is selected as the most frequently occurring one among these outputs. |
| Outcome: | The proposed approach reduces the number of samples required for SC by 9.3x . the majority of tasks can be addressed with only two samples, which has not been possible with prior methods. |
Boosting Self-Consistency with Ranking (2026.acl-srw)
Copied to clipboard
Maria Marina, Daniil Moskovskiy, Sergey Pletenev, Mikhail Salnikov, Alexander Panchenko, Viktor Moskvoretskii
| Challenge: | Existing approaches to improve performance of large language models include self-consistency, RISC, extended reasoning, and iterative self-correction. |
| Approach: | They propose a test-time scaling technique that uses multiple features to score candidate answers in self-consistency as a ranking problem. |
| Outcome: | The proposed method achieves better accuracy-efficiency trade-off than standard self-consistency and strong baselines on three datasets. |
Reasoning Aware Self-Consistency: Leveraging Reasoning Paths for Efficient LLM Sampling (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate reasoning paths before answers, but lack a systematic approach to determine optimal number of samples or select the most faithful rationale. |
| Approach: | They propose a framework that evaluates the quality of reasoning and consistency of answers for each generated sample and uses criteria-based stopping and weighted majority voting to guide early stopping decisions and rationale selection. |
| Outcome: | The proposed framework outperforms existing methods while maintaining accuracy. |
Tracing the Light of Thought: A Probabilistic Self- and Cross-Consistency Verification Mechanism Improving Mathematical Reasoning in LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for evaluating reasoning paths are not efficient, but they are prone to errors. |
| Approach: | They propose a probabilistic self- and cross-consistency framework for mathematical reasoning that employs an accept-reject mechanism to encourage high-quality reasoning paths. |
| Outcome: | The proposed framework improves on 9 LLMs across 4 challenging benchmarks. |
Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregation (2025.findings-acl)
Copied to clipboard
Yiwei Li, Ji Zhang, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Jiayi Shi, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, Kan Li
| Challenge: | Existing studies on self-consistency show that it improves reasoning abilities by aggregating diverse stochastic samples. |
| Approach: | They propose a confidence-driven mechanism that dynamically calibrates temperature to align with high probability modes. |
| Outcome: | The proposed method outperforms fixed-diversity baselines on reasoning tasks and improves both average and best-case performance. |
Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Self-consistency improves reasoning reliability but incurs substantial inference cost . Adaptive self-consistent methods rely on count-based stopping rules that treat all responses equally . |
| Approach: | They propose a method that reframs adaptive sampling from response counting to evidence sufficiency by leveraging response-level confidence. |
| Outcome: | The proposed method reduces inference cost by up to 70% while preserving accuracy on GSM8K. |
Integrate the Essence and Eliminate the Dross: Fine-Grained Self-Consistency for Free-Form Language Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods to improve output quality without aggregating input tokens are limited by the complexity of aggregation of responses. |
| Approach: | They propose to extract and integrate segment-level commonalities from candidate samples to enhance performance of LLMs in open-ended and reasoning tasks. |
| Outcome: | The proposed method improves performance on reasoning, code generation and mathematical reasoning tasks without requiring additional models and overlooking the knowledge present among the candidates. |