Multi-Agent Reasoning Improves Compute Efficiency: Pareto-Optimal Test-Time Scaling (2026.acl-srw)
Copied to clipboard
| Challenge: | Inference methods that prioritize raw performance over cost-effective compute usage are not efficient for real-world applications. |
| Approach: | They evaluate inference scaling strategies to determine their computational efficiency tradeoffs . they find debate and mixture-of-agents outperform self-consistency by 1.3% and 2.7% points . |
| Outcome: | The proposed scaling strategies outperform self-consistency, self-refinement, multi-agent debate and mixture-of-a agents on reasoning tasks. |
Similar Papers
AdaptEvolve: Improving Efficiency of Evolutionary AI Agents through Adaptive Model Selection (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing routing strategies rely on static heuristics or external controllers to optimize performance. |
| Approach: | They propose a framework that leverages intrinsic generation confidence to estimate solvability. |
| Outcome: | Empirical results show that confidence-driven selection yields favorable Pareto frontier . computational cost of state-of-the-art large language models remains a key barrier to scalable deployment . |
Self-Para-Consistency: Improving Reasoning Tasks at Low Cost for Large Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that self-consistency decoding can improve performance for complex reasoning tasks with large language models. |
| Approach: | They propose a self-consistency decoding strategy that generates multiple paraphrases for each test question and then generates reasoning paths for the original and all the paraphrased questions based on greedy decoding. |
| Outcome: | The proposed strategy reduces the sampling number and improves performance on complex reasoning tasks. |
Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations that focus on performance metrics miss a key factor: increased effectiveness due to additional compute. |
| Approach: | They propose to incorporate the compute budget into evaluations to provide a more informative comparison that takes into account both performance metrics and computational cost. |
| Outcome: | The proposed framework outperforms reasoning strategies when they use comparable compute resources. |
What Moves the Pareto Frontier in Tool-Using Agents? A Compute-Aware Study of ReAct Variants (2026.acl-srw)
Copied to clipboard
| Challenge: | Tool-using LLM agents are typically compared by accuracy alone, despite deployments being constrained by inference cost. |
| Approach: | They propose a budgeted evaluation of common strategies for improving ReAct-style tool agents . they compare cumulative accuracy versus token budget on three benchmarks . |
| Outcome: | The proposed methods show that the most common improvement strategies increase inference cost. |
AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios (2026.acl-long)
Copied to clipboard
| Challenge: | brittleness of Large Language Models in reasoningintensive tasks is a problem . current compositional benchmarks focus on *either* commonsense or math reasoning . |
| Approach: | They propose a "Co**mmonsense and Ma**th" benchmark where each compositional task requires a commonsense reasoning step *and* a math reasoning step. |
| Outcome: | The proposed benchmarks show that LLMs can solve both steps in isolation, but their accuracy drops by nearly 30% when the two steps are combined. |
Mixture-of-Minds: Multi-Agent Reinforcement Learning for Table Understanding (2026.acl-long)
Copied to clipboard
Yuhang Zhou, Mingrui Zhang, Ke Li, Mingyi Wang, Qiao Liu, Qifei Wang, Jiayi Liu, Fei Liu, Serena Li, Weiwei LI, Mingze Gao, Abhishek Kumar, Xiangjun Fan, Zhuokai Zhao, Lizhu Zhang
| Challenge: | Large language models (LLMs) have shown promise on understanding and reasoning over tables, but current approaches remain limited. |
| Approach: | They propose a multi-agent framework that decomposes table reasoning into three specialized roles: planning, coding, and answering. |
| Outcome: | The proposed framework decomposes table reasoning into three specialized roles: planning, coding, and answering. |
Foresight Optimization for Strategic Reasoning in Large Language Models (2026.acl-long)
Copied to clipboard
Jessie Wang, Jiawen Duan, Jian Wang, Kaitao Song, Chunpu Xu, Johnny K. W. Ho, YU Fenggang, Johan F. Hoorn, Wenjie Li
| Challenge: | Existing reasoning enhancement methods do not capture foresight in LLMs. |
| Approach: | They propose to integrate opponent modeling principles into policy optimization to enhance strategic reasoning in LLMs by integrating opponent modeling into policy. |
| Outcome: | The proposed method outperforms existing reasoning-based LLMs in out-of-domain scenarios and shows that it significantly enhances strategic reasoning across LLM of varying sizes and origins. |
MAgICoRe: Multi-Agent, Iterative, Coarse-to-Fine Refinement for Reasoning (2025.emnlp-main)
Copied to clipboard
| Challenge: | Excessive refinement can cause over-correction and reduce performance, authors say . they say MAgICoRe is a framework for multi-agent iteration for coarse-to-fine refinement . |
| Approach: | They propose a framework for multi-agent iteration for coarse-to-fine refinement that reduces excessive refinement by categorizing problems as easy or hard. |
| Outcome: | The proposed framework beats Self-Consistency by 3.4%, Best-of-k by 3.2%, and Self-Refine by 4.0% on Llama-3-8B and GPT- 3.5. |
Single-Agent Generation Surpasses Multi-Agent Systems in Semantic Diversity (2026.findings-acl)
Copied to clipboard
| Challenge: | Multi-Agent Systems (MAS) are used to improve reasoning diversity and robustness by simulating interactions among agents with distinct roles. |
| Approach: | They find that a Multi-Output strategy produces the highest diversity without degrading logical validity. |
| Outcome: | The proposed approach outperforms multi-agent systems in semantic diversity . the results point to a more efficient and effective way to expand diversity - the authors say . |
Towards Effective and Efficient Multi-Agent Language Model Systems: Foundations, Prospects, and Applications (2026.acl-tutorials)
Copied to clipboard
| Challenge: | Multi-agent systems powered by large language models still face challenges . tutorial focuses on three core components to build effective and efficient systems . |
| Approach: | This tutorial introduces recent advances in building effective and efficient multi-agent LLM systems . it focuses on three core components: model distillation, dynamic routing, memory- and compute efficient serving . |
| Outcome: | This tutorial introduces state-of-the-art techniques for building efficient and efficient multi-agent LLM systems . it covers coordination and communication among agents, crucial for collective performance . |