PORTS: Preference-Optimized Retrievers for Tool Selection with Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing retrieval-based methods to pre-select tools are often misaligned with tool-calling LLMs due to separate training processes. |
| Approach: | They propose a method to fine-tune retrievers to find useful tools by using a frozen LLM. |
| Outcome: | The proposed method fine-tunes retrievers to find useful tools using a frozen LLM . it improves tool selection accuracy and can be generalized to new queries and tools . |
Similar Papers
ToolDreamer: Instilling LLM Reasoning Into Tool Retrievers (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing retrieval models rank tools based on similarity between query and tool description (TD) Existing tools are not conditioned to learn tool-to-tool relationships (middle). |
| Approach: | They propose a framework that conditions retrieval models to fetch tools based on hypothetical (synthetic) TD generated using an LLM. |
| Outcome: | The proposed framework improves the performance of sparse and dense retrievers with and without training, showcasing its flexibility. |
Improving Tool Retrieval by Leveraging Large Language Models for Query Generation (2025.coling-industry)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown great promise in common sense language understanding, conversational fluency, and reasoning. |
| Approach: | They propose to use Large Language Models to generate a retrieval query and embed it into the prompt to find relevant tools via a nearest-neighbor search. |
| Outcome: | The proposed method improves retrieval for in-domain (seen tools) and out-of-domain settings. |
ToolScope: Enhancing LLM Agent Tool Use through Tool Merging and Context-Aware Filtering (2026.acl-long)
Copied to clipboard
| Challenge: | Large language model (LLM) agents often face strict input context limits, preventing efficient consideration of large toolsets. |
| Approach: | They propose a tool that allows LLMs to merge tools with auto-correction and toolScopeRetriever to rank and select only the most relevant tools for each query. |
| Outcome: | Evaluations on three state-of-the-art LLMs and three open-source tool-use benchmarks show gains of 8.38% to 38.6% in tool selection accuracy. |
Towards Autonomous Tool Utilization in Language Models: A Unified, Efficient and Scalable Framework (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in tool learning for large language models have led to a new trend to allow LLMs to leverage external tools. |
| Approach: | They propose a framework for fine-tuning language models that categorizes queries into three different types . they also introduce an "instruct, execute, and reformat" strategy specifically designed for efficient data annotation . |
| Outcome: | The proposed framework surpasses open-source language models and GPT-3.5/4 on multiple evaluation metrics. |
Retrieval Models Aren’t Tool-Savvy: Benchmarking Tool Retrieval for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) suffer from inherent inabilities to interact with the physical world and access vast, up-to-date knowledge. |
| Approach: | They propose a tool retrieval benchmark for large language models (LLMs) that includes 7.6k diverse retrieval tasks and a corpus of 43k tools. |
| Outcome: | The proposed model performs poorly on the heterogeneous tool retrieval benchmark, resulting in low pass rate and low retrieval quality. |
Enhancing Tool Retrieval with Iterative Feedback from Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods have shown that large language models can handle a certain amount of tools through in-context learning or fine-tuning. |
| Approach: | They propose to enhance tool retrieval with iterative feedback from the large language model by prompting the tool usage model to provide feedback for the tool retriever model in multi-round. |
| Outcome: | The proposed approach achieves advanced performance in both in-domain evaluation and out-of-domain assessment. |
Alignment for Efficient Tool Calling of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in tool learning have enabled large language models to integrate external tools, enhancing their task performance by expanding their knowledge boundaries. |
| Approach: | They propose a framework that combines probabilistic knowledge boundary estimation with dynamic decision-making to allow LLMs to better assess when to invoke tools based on their confidence. |
| Outcome: | The proposed framework shows significant improvements in tool efficiency by reducing unnecessary tool usage. |
ToolExpNet: Optimizing Multi-Tool Selection in LLMs with Similarity and Dependency-Aware Experience Networks (2025.findings-acl)
Copied to clipboard
| Challenge: | Current empirical methods that focus on isolated tools learning struggle with accurate multi-tool selection due to issues like confusing similar tools and neglecting dependencies. |
| Approach: | They propose a tool-learning paradigm which integrates tools and trial-and-error experiences into a network characterized by semantic similarity and dependency relationships. |
| Outcome: | The proposed model outperforms existing methods on multiple real-world API datasets and significantly outperformed baselines. |
Aligning Large Language Models with Recommendation Knowledge (2024.findings-naacl)
Copied to clipboard
Yuwei Cao, Nikhil Mehta, Xinyang Yi, Raghunandan Hulikal Keshavan, Lukasz Heldt, Lichan Hong, Ed Chi, Maheswaran Sathiamoorthy
| Challenge: | Large language models (LLMs) excel at natural language reasoning, but cannot model complex user-item interactions inherent in recommendation tasks. |
| Approach: | They propose to equip large language models with recommendation-specific knowledge to address this gap by combining Masked Item Modeling and Bayesian Personalized Ranking (BPR) auxiliary task data samples are generated that encode item correlations and user preferences. |
| Outcome: | Experiments on Amazon Toys & Games, Beauty, and Sports & Outdoors show that the proposed method outperforms conventional and LLM-based baselines by significant margins in retrieval. |
ToolSpectrum: Towards Personalized Tool Utilization for Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches focus on functional tool selection following user instructions while overlooking the critical role of context-aware personalization in tool selection. |
| Approach: | They propose a benchmark to evaluate LLMs’ capabilities in personalized tool utilization. |
| Outcome: | The proposed benchmark evaluates LLMs' capabilities in personalized tool utilization. |