Demonstration Selection Strategies for Numerical Time Series Data-to-Text (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Demonstration selection is a critical step in in-context learning, where a prompt is fed into large language models. |
| Approach: | They propose to use sequence similarity-based selection and task-specific knowledge-based demonstration selection methods to select similar instances from an example bank. |
| Outcome: | The proposed methods outperform baseline selections and often surpass fine-tuned models on two benchmark datasets and human judges confirm their performance. |
Similar Papers
Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to select demonstration examples for in-context learning are based on token embeddings. |
| Approach: | They propose an algorithm to select demonstration examples for in-context learning of a query set . they use gradients of the output taken in the input embedding space to estimate model outputs . |
| Outcome: | The proposed algorithm outperforms existing methods based on token embeddings by 11% . it scales up subset selection that would otherwise run full inference by 37.7 on models with 34 billion parameters . |
Revisiting Demonstration Selection Strategies in In-Context Learning (2024.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown an impressive ability to perform a wide range of tasks using in-context learning (ICL). |
| Approach: | They propose a data- and model-dependent method to select models using in-context learning, TopK + ConE, and propose unified explanations for the effectiveness of previous methods. |
| Outcome: | The proposed method improves language understanding and generation tasks with different model scales. |
Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Recent learning-based demonstration selection methods have proven beneficial to in-context learning (ICL) by choosing more useful exemplars. |
| Approach: | They propose two methods to capture task-agnostic similarities between input and output of LLMs. |
| Outcome: | The proposed methods integrate task-agnostic similarities of different levels between input and output of exemplars and test cases to eliminate costly data collection. |
In-Context Learning with Iterative Demonstration Selection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing literature has highlighted the importance of selecting examples that are diverse or semantically similar to the test sample . Existing studies have shown that the optimal selection dimension, i.e., diversity or similarity, is task-specific. |
| Approach: | They propose to use zero-shot chain-of-thought reasoning to iteratively select examples that are diverse but still strongly correlated with the test sample as ICL demonstrations. |
| Outcome: | The proposed method outperforms existing demonstration selection methods on reasoning, question answering, and topic classification tasks. |
Representative Demonstration Selection for In-Context Learning with Two-Stage Determinantal Point Process (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods tend to select different demonstrations for each test instance, which is time-consuming and poses limitations in practical scenarios. |
| Approach: | They propose to select a representative subset of in-context demonstrations that can prompt different test instances in a specific task. |
| Outcome: | The proposed method can be used to generate representative in-context demonstrations. |
Active Example Selection for In-Context Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning performance is unstable across samples of examples, suggesting the idiosyncrasies of how language models acquire information. |
| Approach: | They propose a reinforcement learning algorithm for identifying generalizable policies to select demonstration examples and propose 'in-context learning' performance can be highly unstable across samples of examples, suggesting the idiosyncrasies of how language models acquire information. |
| Outcome: | The proposed model can perform tasks with examples with a 5.8% improvement on GPT-2 and GPT-3, but the improvement diminishes on larger models, suggesting emerging capabilities of large language models. |
Se2: Sequential Example Selection for In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Prior work has explored the selection of examples for in-context learning, neglecting the internal relationships between examples and exist an inconsistency between training and inference. |
| Approach: | They propose a sequential-aware method that leverages the LLM’s feedback on varying context, aiding in capturing inter-relationships and sequential information among examples. |
| Outcome: | Experiments on 23 NLP tasks show that Se2 surpasses baselines and achieves 42% relative improvement over random selection. |
Good Examples Make A Faster Learner: Simple Demonstration-based Learning for Low-resource NER (2022.acl-long)
Copied to clipboard
Dong-Ho Lee, Akshen Kadakia, Kangmin Tan, Mahak Agarwal, Xinyu Feng, Takashi Shibuya, Ryosuke Mitani, Toshiyuki Sekiya, Jay Pujara, Xiang Ren
| Challenge: | Recent advances in prompt-based learning have shown strong results on few-shot text classification by using cloze-style templates. |
| Approach: | They propose a demonstration-based learning method which lets the input be prefaced by task demonstrations for in-context learning. |
| Outcome: | The proposed method improves on in-domain learning and domain adaptation in low-resource settings. |
Contrastive Demonstration Tuning for Pre-trained Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies focus on searching discrete or continuous prompts or optimized verbalizers, yet the demonstration examples are crucial for an excellent final performance of prompt-tuning. |
| Approach: | They propose a pluggable, extensible, and efficient approach to prompt tuning that is free of demonstration sampling. |
| Outcome: | The proposed approach can be pluggable, extensible, and efficient on 16 datasets. |
Let Me Check the Examples: Enhancing Demonstration Learning via Explicit Imitation (2023.acl-short)
Copied to clipboard
| Challenge: | Existing work only concatenates answered examples as demonstrations to prompt template without any additional operation, neglecting the prompt-demonstration dependencies. |
| Approach: | They propose to concatenate answered examples as demonstrations to prompt template without any additional operation, neglecting the prompt-demonstration dependencies. |
| Outcome: | Experiments show that the proposed method achieves state-of-the-art performance on 5 out of 14 classification corpus. |