Chaoyang Wang, Kaituo Feng, Dongyang Chen, Zhongyu Wang, Zhixun Li, Sicheng Gao, Meng Meng, Xu Zhou, Manyuan Zhang, Yuzhang Shang, Xiangyu Yue
| Challenge: | Existing models exhibit blind tool-use reasoning patterns, which significantly increases inference overhead and degrades model performance. |
| Approach: | They propose an MLLM that performs adaptive tool-use by determining whether a visual problem truly requires tools. |
| Outcome: | The proposed model outperforms existing methods in visual reasoning tasks. |
Similar Papers
EMTIR-GRPO: Efficient Multi-Tool Augmented Large Language Models via Reinforcement Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Tool-integrated reasoning (TIR) enables large language models to invoke external tools for tasks beyond their internal capacity but often suffers from tool overuse. |
| Approach: | They propose an algorithm that uses a composite reward to model tool costs and tool efficiency. |
| Outcome: | The proposed algorithm models heterogeneous tool costs and encourages more cost-effective tool-use strategies. |
AdaV: Adaptive Text-visual Redirection for Vision-Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Vision-language models often generate excessive visual tokens, leading to poor performance . a novel training-free visual token pruning method is proposed to improve performance despite the computational cost associated with VLMs. |
| Approach: | They propose a training-free visual token pruning method that reduces biased token pruning . they plan to open-source the code upon publication . |
| Outcome: | The proposed method reduces biased token pruning and enhances model robustness with limited visual token budget. |
PruneVid: Visual Token Pruning for Efficient Video Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to video token pruning face significant computational challenges due to the redundancy inherent in video data. |
| Approach: | They propose a training-free visual token pruning method that reduces the redundancy inherent in video data and leverages LLMs’ inherent ability to selectively prune visual tokens irrelevant to specific queries. |
| Outcome: | The proposed method can prune over 80% of tokens while maintaining competitive performance when combined with different video LLMs. |
MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Prompting (2023.acl-short)
Copied to clipboard
| Challenge: | Recent studies have focused on using a single external tool to solve a problem with large language models and have not addressed different problems together. |
| Approach: | They propose a framework that leverages chain-of-thought prompting to incorporate multiple external tools into the reasoning process. |
| Outcome: | The proposed framework outperforms baselines and achieves state-of-the-art performance on a task that requires both numerical reasoning and domain-specific knowledge. |
MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scale (2025.acl-long)
Copied to clipboard
Jiawei Guo, Tianyu Zheng, Yizhi Li, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Graham Neubig, Wenhu Chen, Xiang Yue
| Challenge: | Current instruction-tuning datasets focus on simplistic visual question answering tasks, and provide phrase-level answers without any intermediate rationales. |
| Approach: | They propose to use open-source multimodal large language models to train MLLMs on a dataset with 12M instruction-response pairs to elicit CoT reasoning. |
| Outcome: | The proposed model achieves state-of-the-art performance on benchmarks such as MathVerse, MMMU-Pro, and MuirBench, and gains improvements of up to 4% on non-reasoning-based benchmarks. |
Simple-VGC: Enhancing Visual Grounding in Multimodal Reasoning via Adaptive Tool Composition (2026.acl-long)
Copied to clipboard
| Challenge: | Existing multimodal large language models suffer from systematic failures in basic visual understanding. |
| Approach: | They propose a tool-augmented reasoning framework with three targeted compensation strategies to address these problems. |
| Outcome: | The proposed framework improves visual grounding by re-injecting the original image to mitigate visual forgetting, the authors show . the proposed framework also improves the accuracy of the visual inputs, the researchers show - and the results are promising . |
AdaReTaKe: Adaptive Redundancy Reduction to Perceive Longer for Video-language Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Multimodal Large Language Models (MLLMs) are limited by context length when processing long videos. |
| Approach: | They propose a training-free method that flexibly reduces redundancy by allocating compression ratios among time and model layers with theoretical guarantees. |
| Outcome: | Experiments on videoMME, MLVU, LongVideoBench, and LVBench show that AdaRETAKE outperforms existing methods by 2.3% and 2.8% for 7B and 72B models. |
Mitigating Visual Forgetting via Take-along Visual Conditioning for Multi-modal Long CoT Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Large Language Models (LLMs) have demonstrated enhanced reasoning capabilities, evolving from simple Chain-of-Thought (CoT) prompting to advanced, product-oriented solutions like OpenAI o1 . |
| Approach: | They propose a strategy that shifts image input to critical reasoning stages and compresses redundant visual tokens via dynamic pruning. |
| Outcome: | The proposed model achieves state-of-the-art on five mathematical reasoning benchmarks (+3.4% vs previous sota) and demonstrates iterative reasoning capabilities for complex multi-step tasks. |
A Comprehensive Evaluation of Tool-Assisted Generation Strategies (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Various few-shot tool-usage strategies have been proposed to overcome LMs' shortcomings. |
| Approach: | They propose to augment language models with tools to overcome their shortcomings . they find strong no-tool baselines are competitive to tool-assisted strategies . |
| Outcome: | The proposed strategies outperform those that refine incorrect outputs with tools in knowledge-retrieval tasks, the study finds . the findings suggest few-shot tool integration is still an open challenge . |
API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs (2023.emnlp-main)
Copied to clipboard
Minghao Li, Yingxiu Zhao, Bowen Yu, Feifan Song, Hangyu Li, Haiyang Yu, Zhoujun Li, Fei Huang, Yongbin Li
| Challenge: | Recent research has demonstrated that Large Language Models (LLMs) can enhance their capabilities by utilizing external tools. |
| Approach: | They propose a runnable evaluation system consisting of 73 API tools and an annotation system for 314 tool-use dialogues with 753 API calls. |
| Outcome: | The proposed benchmark assesses the effectiveness of existing LLMs by analyzing 314 tool-use dialogues with 753 API calls. |