The Geometry of Prompting: Unveiling Distinct Mechanisms of Task Adaptation in Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have focused on the inputoutput behavior of LMs, leaving the internal mechanisms behind ICL largely unexplored. |
| Approach: | They investigate how different prompting methods modify internal representations in pre-trained language models. |
| Outcome: | The proposed model can be programmed with natural language to perform a wide array of tasks without expensive fine-tuning. |
Similar Papers
Tree Prompting: Efficient Task Adaptation without Fine-Tuning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Pretrained language models (LMs) are the main interface for applying them to new tasks, but their large size makes them difficult to fine-tune with gradients for specific downstream tasks. |
| Approach: | They propose to use training data to form a decision tree based on prompt-LM calls, with each prompt determined by the outcomes of previous calls. |
| Outcome: | The proposed method improves accuracy over competing methods and is competitive with fine-tuning. |
Reframing Instructional Prompts to GPTk’s Language (2022.findings-acl)
Copied to clipboard
| Challenge: | Using reframing techniques, we find that instructional prompts are easier to follow for Language Models (LMs) |
| Approach: | They propose reframing techniques for manual reformulation of prompts into more effective ones . they compare performance of LMs prompted with reframed instructions on 12 NLP tasks . |
| Outcome: | The reframing techniques used for prompt reformulation improve performance on 12 tasks . the techniques boost performance on LMs with different sizes compared with original prompts . |
The language of prompting: What linguistic properties make a prompt successful? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that pretraining and instruction-tuned LLMs can achieve impressive performance on a multitude of tasks. |
| Approach: | They propose to use a standard for prompting research to better understand linguistic properties of LLMs. |
| Outcome: | The proposed standard would improve the performance of pre-trained and instruction-tuned LLMs on a multitude of tasks. |
Revisiting Automated Prompting: Are We Actually Doing Better? (2023.acl-short)
Copied to clipboard
| Challenge: | Recent work demonstrates that Large Language Models are great few-shot learners, and prompting significantly increases their performance on a range of downstream tasks. |
| Approach: | They revisit techniques for automated prompting on six different downstream tasks and a larger range of K-shot learning settings. |
| Outcome: | The proposed approach outperforms manual prompting on six different downstream tasks and a larger range of K-shot learning settings. |
Prompting Language Models for Linguistic Structure (2023.acl-long)
Copied to clipboard
| Challenge: | Existing prompting methods can test this hypothesis on autoregressive PLMs. |
| Approach: | They propose a structured prompting approach for linguistic structured prediction tasks that performs zero- and few-shot sequence tagging with autoregressive PLMs. |
| Outcome: | The proposed approach shows that the model can perform few-shot sequence tagging on part-of-speech taging, named entity recognition, and sentence chunking tasks. |
The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are used to label data across many domains and for myriad tasks. |
| Approach: | They ask large language models to label data using a series of decisions by practitioners . they find that even the smallest perturbations can change the LLM's answer . |
| Outcome: | The proposed model can be used to quickly get a response for arbitrary tasks. |
Revealing the Unwritten: Visual Investigation of Beam Search Trees to Address Language Model Prompting Challenges (2025.acl-demo)
Copied to clipboard
Thilo Spinner, Rita Sevastjanova, Rebecca Kehlbeck, Tobias Stähle, Daniel A. Keim, Oliver Deussen, Andreas Spitz, Mennatallah El-Assady
| Challenge: | generative language models have become popular, but comprehending their outputs remains challenging for NLP practitioners and linguistic experts. |
| Approach: | They propose to use a beam search tree to examine model outputs to provide information on runner-up candidates and their corresponding probabilities to address these challenges. |
| Outcome: | The proposed method validates existing results and offers additional insights. |
Unleashing the Power of Large Language Models in Zero-shot Relation Extraction via Self-Prompting (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for zero-shot Relation Extraction (RE) lack detailed, context-specific prompts for understanding various sentences and relations. |
| Approach: | They propose a framework that uses a three-stage diversity approach to prompt LLMs by generating multiple synthetic samples that encapsulate specific relations from scratch. |
| Outcome: | The proposed framework outperforms existing LLM-based zero-shot RE methods on benchmark datasets and shows that it produces high-quality synthetic data that enhances performance. |
Beyond Prompt Engineering: A Systematic Analysis of Prompt Lexical Sensitivity and Its Impacts on Quality (2026.findings-acl)
Copied to clipboard
Qipeng Xie, Zi Liang, Jiafei Wu, Yufei Chen, Weizheng Wang, Wenao Ma, Zhong Ming, Haiqin Yang, Kaishun Wu
| Challenge: | Existing studies on prompt engineering have focused on optimizing models for performance under stylistic perturbations. |
| Approach: | They conduct the first analysis of n-gram token-level mechanisms . they find that higher average performance is inherently associated with lower variance and greater stability. |
| Outcome: | The proposed model reduces the variance of the generated code by 40% . the proposed model is based on a large-scale dataset of 132,000 prompt variants . |
No One Fits All: From Fixed Prompting to Learned Routing in Multilingual LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies show that translation-based prompting is not universally optimal for multilingual LLMs. |
| Approach: | They evaluate translation-based prompting across ten languages and four benchmarks . they propose a lightweight classifier that predicts whether native or translation- based prompts are optimal . |
| Outcome: | The proposed classifiers achieve statistically significant improvements over fixed prompting strategies across ten languages and four benchmarks. |