Wenxuan Wang, Shi Juluan, Zixuan Ling, Yuk-Kit Chan, Chaozheng Wang, Cheryl Lee, Youliang Yuan, Jen-tse Huang, Wenxiang Jiao, Michael R. Lyu
| Challenge: | Despite their impressive capabilities, LLMs struggle with complex computations and delivering accurate, timely information. |
| Approach: | They propose a framework that prompts LLM agents to ask questions when they encounter obstacles due to unclear instructions and an automated evaluation tool called ToolEvaluator. |
| Outcome: | The proposed framework outperforms existing frameworks for tool learning in the Noisy ToolBench. |
Similar Papers
Structured Uncertainty guided Clarification for LLM Agents (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to clarifying tasks fail when user instructions are ambiguous or incomplete. |
| Approach: | They propose a principled formulation of structured uncertainty that operates directly over tool parameters and their domains. |
| Outcome: | The proposed framework improves when2call accuracy and training-time sample efficiency. |
A Joint Optimization Framework for Enhancing Efficiency of Tool Utilization in LLM Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts for tool utilization involve an LLM agent that contains instructions on using the description of the available tools to determine and call the tools required to solve the problem. |
| Approach: | They propose to optimize the context of LLM agents by combining the instructions provided in agent prompts and tool descriptions to enhance their interaction. |
| Outcome: | The proposed framework improves both the instructions provided in agent prompt and tool description, enhancing their interaction. |
ACEBench: A Comprehensive Evaluation of LLM Tool Usage (2025.findings-emnlp)
Copied to clipboard
Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Yuefeng Huang, Xiangcheng Liu, Wang Xinzhi, Wu Liu
| Challenge: | Existing benchmarks for evaluating LLMs’ tool usage face several limitations: limited evaluation scenarios, lacking assessments in real multi-turn dialogue contexts; narrow evaluation dimensions, with insufficient detailed assessments of how LLM use tools; and reliance on LLM or real API executions for evaluation, which introduces significant overhead. |
| Approach: | ACEBench is a benchmark for evaluating tool usage in Large Language Models . it categorizes data into three primary types based on evaluation methodology: Normal, Special, and Agent. |
| Outcome: | ACEBench categorizes data into three primary types based on evaluation methodology: Normal, Special, and Agent. |
ToolCPT: Improving Tool Utilization in LLM Agents via Continuous Pre-training (2026.findings-acl)
Copied to clipboard
| Challenge: | Current approaches to enhancing tool use for LLM-based agents focus on post-training fine-tuning or test-time context extension. |
| Approach: | They propose to enhance tool knowledge for LLM-based agents during continuous pre-training . they curate 5.1 million code artifacts from large-scale, high-quality code repositories . |
| Outcome: | The proposed model outperforms existing methods on out-of-distribution tools on multiple benchmarks. |
Small LLMs Are Weak Tool Learners: A Multi-LLM Agent (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have revolutionized natural language processing with impressive capabilities, but they lack domain specificity, real-time information and face challenges in solving specialized problems. |
| Approach: | They propose a multi-LLM approach that decomposes the aforementioned capabilities into a planner, caller, and summarizer. |
| Outcome: | The proposed model outperforms existing models by demonstrating its effectiveness and advantages in tool learning. |
When and What to Ask: AskBench and Rubric-Guided RLVR for LLM Clarification (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models respond even when prompts omit critical details or include misleading information, leading to hallucinations or reinforced misconceptions. |
| Approach: | They propose an interactive benchmark that converts standard QA pairs into multi-turn interactions with explicit checkpoints. |
| Outcome: | The proposed benchmark improves accuracy, rubric adherence, and interaction efficiency with strong generalization to unseen domains. |
CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty (2026.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for Large Language Model (LLM) agents focus on task completion under idealistic settings but overlook reliability in real-world, user-facing applications. |
| Approach: | They propose a benchmark to evaluate consistency, uncertainty handling, and capability awareness in multi-turn, tool-using LLM agents in an in-car assistant domain. |
| Outcome: | The proposed benchmarks evaluate consistency, uncertainty handling, and capability awareness in multi-turn, tool-using LLM agents in an in-car assistant domain. |
Tools Fail: Detecting Silent Errors in Faulty Tools (2024.emnlp-main)
Copied to clipboard
| Challenge: | a failure in one tool can trigger a cascade of errors, leading to complete task failure. |
| Approach: | They propose a framework for tools more broadly which explores a model’s ability to detect “silent” tool errors and reflect on how to plan. |
| Outcome: | The proposed approach shows that the model can detect "silent" tool errors and plan. |
How Can Input Reformulation Improve Tool Usage Accuracy in a Complex Dynamic Environment? A Study on tau-bench (2025.findings-emnlp)
Copied to clipboard
Venkatesh Mishra, Amir Saeidi, Satyam Raj, Mutsumi Nakamura, Gaowen Liu, Ali Payani, Jayanth Srinivasa, Chitta Baral
| Challenge: | Recent advances in reasoning and planning capabilities of large language models have enabled their potential as autonomous agents capable of tool use in dynamic environments. |
| Approach: | They propose an input-reformulation multi-agent framework that reformulates user queries . |
| Outcome: | The proposed framework outperforms ReAct, Function Calling, and Self-Reflection in overall pass5 scores. |
Learning API Functionality from In-Context Demonstrations for Tool-based Agents (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Documentation is often missing, outdated, privatized, or inconsistent in tool-based agents. |
| Approach: | They propose a task that allows agents to learn API functionality directly from in-context demonstrations. |
| Outcome: | The proposed model improves the task success rate of an API-based agent by providing explicit function calls and natural language critiques. |