PLAY2PROMPT: Zero-shot Tool Instruction Optimization for LLM Agents via Tool Play (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing solutions for large language models rely on manual rewriting or labeled data for validation . Existing approaches rely only on comprehensive tool documentation and in-context demonstrations . |
| Approach: | They propose a framework that "plays" with each tool to explore its input-output behaviors. |
| Outcome: | Experiments show that PLAY2PROMPT improves zero-shot tool performance across open and closed models. |
Similar Papers
LLMs Are Zero-Shot Context-Aware Simultaneous Translators (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing SiMT systems operate on a sentence level, disregarding the context established by previous sentences or the broader context implied by previous words. |
| Approach: | They show that open-source LLMs perform on par with or better than some state-of-the-art baselines in simultaneous machine translation tasks, zero-shot. |
| Outcome: | The proposed models perform on par with or better than state-of-the-art baselines in simultaneous machine translation tasks, zero-shot. |
A Joint Optimization Framework for Enhancing Efficiency of Tool Utilization in LLM Agents (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing efforts for tool utilization involve an LLM agent that contains instructions on using the description of the available tools to determine and call the tools required to solve the problem. |
| Approach: | They propose to optimize the context of LLM agents by combining the instructions provided in agent prompts and tool descriptions to enhance their interaction. |
| Outcome: | The proposed framework improves both the instructions provided in agent prompt and tool description, enhancing their interaction. |
Wav2Prompt: End-to-End Speech Prompt Learning and Task-based Fine-tuning for Text-based LLMs (2025.naacl-long)
Copied to clipboard
| Challenge: | Text-based large language models (LLMs) can be applied to a wide range of tasks without being explicitly trained. |
| Approach: | They propose a method which integrates spoken input with a text-based large language model (LLM) it takes LLM token embeddings as training targets and utilises a continuous integrate-and-fire mechanism for explicit speech-text alignment. |
| Outcome: | The proposed model can be applied to speech translation, speech understanding and spoken-query-based question answering tasks. |
Better Zero-Shot Reasoning with Role-Play Prompting (2024.naacl-long)
Copied to clipboard
Aobo Kong, Shiwan Zhao, Hao Chen, Qicheng Li, Yong Qin, Ruiqi Sun, Xin Zhou, Enzhi Wang, Xiaohang Dong
| Challenge: | Recent years have witnessed a paradigm shift in natural language processing, driven by large language models such as GPT-3, PaLM, and Llama. |
| Approach: | They propose a strategy for role-play prompting and assess its performance under the zero-shot setting. |
| Outcome: | The proposed method outperforms the standard zero-shot prompting approach across 12 reasoning benchmarks. |
Revisiting LLMs as Zero-Shot Time Series Forecasters: Small Noise Can Break Large Models (2025.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown remarkable performance across diverse tasks without domain-specific training, fueling interest in their potential for time series forecasting. |
| Approach: | They evaluate the effectiveness of LLMs as zero-shot forecasters compared to state-of-the-art domain-specific models by encoding sequences directly within prompts. |
| Outcome: | The proposed models perform well across multiple domains while reducing the need for domain-specific training. |
Zero-Shot Text Classification with Self-Training (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large pretrained language models have increased attention to zero-shot text classification. |
| Approach: | They propose a plug-and-play method to bridge this gap by requiring only class names along with an unlabeled dataset. |
| Outcome: | The proposed model can be trained on a natural language inference dataset and performs on dozens of unseen tasks without the need for domain expertise or trial and error. |
ZEROTOP: Zero-Shot Task-Oriented Semantic Parsing using Large Language Models (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing LLMs cannot generalize to domain-specific parsing tasks in a zero-shot setting. |
| Approach: | They propose a task-oriented parsing method that decomposes parse problem into abstractive and extractive question-answering problems. |
| Outcome: | The proposed method decomposes a parsing problem into abstractive and extractive question-answering (QA) problems. |
Towards Tool Use Alignment of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies on tool use with LLMs focus on enhancing tool-calling ability of LLM . e.g., LLM should not answer unsafe tool use relevant instructions or insecure tool responses to ensure reliability and harmlessness. |
| Approach: | They propose to use supervised fine-tuning and preference learning to align LLMs with H2A principle for tool use. |
| Outcome: | The proposed model demonstrates that LLMs can generate truthful and helpful responses while remaining harmless. |
ACEBench: A Comprehensive Evaluation of LLM Tool Usage (2025.findings-emnlp)
Copied to clipboard
Chen Chen, Xinlong Hao, Weiwen Liu, Xu Huang, Xingshan Zeng, Shuai Yu, Dexun Li, Yuefeng Huang, Xiangcheng Liu, Wang Xinzhi, Wu Liu
| Challenge: | Existing benchmarks for evaluating LLMs’ tool usage face several limitations: limited evaluation scenarios, lacking assessments in real multi-turn dialogue contexts; narrow evaluation dimensions, with insufficient detailed assessments of how LLM use tools; and reliance on LLM or real API executions for evaluation, which introduces significant overhead. |
| Approach: | ACEBench is a benchmark for evaluating tool usage in Large Language Models . it categorizes data into three primary types based on evaluation methodology: Normal, Special, and Agent. |
| Outcome: | ACEBench categorizes data into three primary types based on evaluation methodology: Normal, Special, and Agent. |
On Zero-Shot Counterspeech Generation by LLMs (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used in numerous NLP tasks, including counterspeech generation. |
| Approach: | They propose three different prompting strategies for generating different types of counterspeech and propose a set of prompting techniques for counterspeak generation. |
| Outcome: | The proposed prompting strategies improve the performance of the models for counterspeech generation in two datasets, but with high toxicity with increase in model size. |