Collaborative Beam Search: Enhancing LLM Reasoning via Collective Consensus (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to improve the reasoning capabilities of large language models (LLMs) depend on domain-specific external verifiers or self-evaluation which is brittle and prompt-sensitive. |
| Approach: | They propose a framework that harnesses the collective intelligence of multiple large language models across both generation and verification stages. |
| Outcome: | The proposed framework outperforms singlemodel scaling and multi-model ensemble baselines on six tasks by over 4 percentage points in average accuracy. |
Similar Papers
Allies: Prompting Large Language Model with Beam Search (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to build LLMs with stacking are limited by their information coverage and low fault tolerance. |
| Approach: | They propose a method that leverages large language models to iteratively generate new queries from an input query. |
| Outcome: | The proposed method outperforms baselines on open-domain question answering benchmarks. |
FiDeLiS: Faithful Reasoning in Large Language Models for Knowledge Graph Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing retrieval-based or agent-based methods are prone to generating erroneous or hallucinated outputs. |
| Approach: | They propose a framework to leverage knowledge graphs as external knowledge sources to improve the factuality of LLM responses by anchoring answers to verifiable reasoning steps retrieved from KGs. |
| Outcome: | The proposed framework improves factuality and interpretability across benchmarks and reduces computational costs. |
Multi-LLM Collaborative Search for Complex Problem Solving (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) often struggle with complex reasoning tasks due to the vast reasoning space inherent in the complexity and inherent ambiguities of natural languages. |
| Approach: | They propose a mixture-of-search-agents paradigm that integrates diverse reasoning pathways by combining independent exploration and iterative refinement among multiple LLMs. |
| Outcome: | The proposed approach improves performance over single-agent and multi-agend baselines in complex mathematical and commonsense reasoning tasks. |
Examining Inter-Consistency of Large Language Models Collaboration: An In-depth Analysis via Debate (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies focus on inconsistency issues within a single LLM, while we explore the inter-consistencies among multiple LLMs for collaboration. |
| Approach: | They propose a formal debate framework to examine whether LLMs can collaborate effectively to achieve a consensus for a shared goal. |
| Outcome: | The proposed framework enables LLMs to achieve consensus in three real-world debate scenarios with real-time scenarios aligned to the LLM's goals. |
ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) still struggle with natural language reasoning tasks. |
| Approach: | They propose a multi-model multi-agent framework that enhances collaborative reasoning between LLM agents via multiple rounds of discussion. |
| Outcome: | Experiments on seven benchmarks show that ReConcile significantly improves LLM reasoning on individual and team levels. |
BeamAggR: Beam Aggregation Reasoning over Multi-source Knowledge for Multi-hop Question Answering (2024.acl-long)
Copied to clipboard
Zheng Chu, Jingchang Chen, Qianglong Chen, Haotian Wang, Kun Zhu, Xiyuan Du, Weijiang Yu, Ming Liu, Bing Qin
| Challenge: | Large language models (LLMs) have demonstrated strong reasoning capabilities, but they still suffer from factual errors when tackling knowledge-intensive tasks. |
| Approach: | They propose a reasoning framework for knowledge-intensive multi-hop QA that prioritizes promising answers at each hop of question. |
| Outcome: | The proposed framework outperforms SOTA methods on four open-domain multi-hop reasoning datasets by 8.5%. |
Harnessing Consistency for Robust Test-Time LLM Ensemble (2026.findings-eacl)
Copied to clipboard
Zhichen Zeng, Qi Yu, Xiao Lin, Ruizhong Qiu, Xuying Ning, Tianxin Wei, Yuchen Yan, Jingrui He, Hanghang Tong
| Challenge: | Existing efforts to improve LLM ensemble quality have focused on model consistency, but failures are often due to heterogeneous tokenization schemes and varying model expertise. |
| Approach: | They propose a plug-and-play technique that harnesses model consistency for robust LLM ensemble. |
| Outcome: | The proposed technique improves ensemble performance and robustness against erroneous signals. |
An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Making (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have sparked interest in collaborative LLM agents. |
| Approach: | They propose to integrate various ordinal preferential voting mechanisms into LLMs to improve reasoning capabilities and robustness. |
| Outcome: | The proposed method improves reasoning capabilities and robustness of leading LLMs without complex system designs. |
ConSensus: Multi-Agent Collaboration for Multimodal Sensing (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models are increasingly grounded in sensor data to perceive and reason about human physiology and the physical world. |
| Approach: | They propose a training-free multi-agent collaboration framework that decomposes multimodal sensing tasks into specialized, modality-aware agents. |
| Outcome: | The proposed framework matches or exceeds debate methods on multimodal sensing benchmarks while achieving 12.7 times reduction in token cost. |
DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text (2024.findings-naacl)
Copied to clipboard
| Challenge: | Retrievalaugmented LLMs have been used to ground LLM in external knowledge . a gap exists in the current landscape regarding the effectiveness of grounding LLM on heterogeneous knowledge sources. |
| Approach: | They propose a model that uses symbolic language to generate symbolic queries . they use a dataset that is generated using predefined reasoning chains and human annotation . |
| Outcome: | The proposed model outperforms previous approaches by a significant margin in QA tasks over text. |