Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Using a Llama-3.2-3B-Instruct backbone, we evaluate four adaptation mechanisms across four benchmarks: Gorilla APIBench, Spider 2.0, WebArena, and InterCode. |
| Approach: | They compare hypernetwork-based LoRA adaptation against carefully designed few-shot prompting in a controlled experiment . they find that few- shot prompting contributes +21.5% to performance and documentation contributes 0% . |
| Outcome: | The hypernetwork-based LoRA adaptation provides no measurable improvement over few-shot prompting alone. |
Similar Papers
Making Pre-trained Language Models Better Few-shot Learners (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that the GPT-3 model can perform few-shots on language understanding tasks with a natural-language prompt and a few task demonstrations. |
| Approach: | They propose a technique for fine-tuning language models using a few examples . they propose LM-BFF, which uses prompt-based fine-uning and a pipeline for automating prompt generation . |
| Outcome: | The proposed approach outperforms standard fine-tuning procedures on a range of NLP tasks. |
A Comprehensive Evaluation of Tool-Assisted Generation Strategies (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Various few-shot tool-usage strategies have been proposed to overcome LMs' shortcomings. |
| Approach: | They propose to augment language models with tools to overcome their shortcomings . they find strong no-tool baselines are competitive to tool-assisted strategies . |
| Outcome: | The proposed strategies outperform those that refine incorrect outputs with tools in knowledge-retrieval tasks, the study finds . the findings suggest few-shot tool integration is still an open challenge . |
It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners (2021.naacl-main)
Copied to clipboard
| Challenge: | Pretraining ever-larger language models on massive corpora requires enormous amounts of compute. |
| Approach: | They propose to convert textual inputs into cloze questions that contain a task description . they also exploit unlabeled data to improve their performance . |
| Outcome: | The proposed model outperforms GPT-3 with PET/iPET with cloze questions and unlabeled data. |
RIFF: Learning to Rephrase Inputs for Few-shot Fine-tuning of Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained Language Models (PLMs) can be fine-tuned for downstream text processing tasks. |
| Approach: | They propose to use paraphrases to enrich the input text of a few-shot model with a Maximum-Marginal Likelihood objective to improve performance. |
| Outcome: | The proposed methods improve performance beyond what can be achieved with parameter-efficient fine-tuning alone. |
Few-shot Adaptation Works with UnpredicTable Data (2023.acl-long)
Copied to clipboard
| Challenge: | Prior work on language models (LMs) shows that training on a large number of diverse tasks improves few-shot learning (FSL) performance on new tasks. |
| Approach: | They finetuned 413,299 tasks from internet tables to find narrow subsets outperform more diverse datasets. |
| Outcome: | The proposed model outperforms training on 40 human-curated NLP datasets on 52 downstream tasks, but not proportionally to dataset scale. |
Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models (2022.findings-acl)
Copied to clipboard
| Challenge: | Prompting language models (LMs) with training examples and task descriptions has been seen as critical to recent successes in few-shot learning. |
| Approach: | They propose to fine tune masked language models with training examples and task descriptions to reduce prompt engineering by using null prompts. |
| Outcome: | The proposed prompts can be used to improve few-shot learning by finetuning only the bias terms while updating only 0.1% of the parameters. |
Systematic Analysis for Pretrained Language Model Priming for Parameter-Efficient Fine-tuning (2024.naacl-srw)
Copied to clipboard
| Challenge: | Parameter-efficient (PE) methods for adapting pre-trained language models to downstream tasks are still lacking in many cases. |
| Approach: | They propose a general PE priming framework to enhance few-shot adaptation and generalization ability of PE methods. |
| Outcome: | The proposed framework reveals that the best priming strategy facilitates adaptation to target tasks. |
Small Language Models Are Good Too: An Empirical Study of Zero-Shot Classification (2024.lrec-main)
Copied to clipboard
| Challenge: | Using small language models, we challenge the dominance of large models in text classification by prompting. |
| Approach: | They compare the performance of small and large language models in a zero-shot context using different architectures and scoring functions. |
| Outcome: | The proposed model outperforms large models in a zero-shot context. |
PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from related Example Banks (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated impressive few-shot learning capabilities through in-context learning. |
| Approach: | They propose a novel Alternating Minimization approach for example selection that improves ICL performance on low-resource Indic languages. |
| Outcome: | The proposed approach outperforms existing frameworks for retrieving examples on low-resource Indic languages. |
Instances Need More Care: Rewriting Prompts for Instances with LLMs in the Loop Yields Better Zero-Shot Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized zero-shot task performance, mitigating the need for task-specific annotations while enhancing task generalizability. |
| Approach: | They propose an approach that optimizes the zero-shot prompts for individual task instances following an innovative manner of "LLMs in the loop" their results show that PRomPTed outperforms naive zero- shot approaches and a strong baseline which refines the task output instead of the input prompt. |
| Outcome: | The proposed approach outperforms naive approaches and a strong baseline which refines the task output instead of the input prompt. |