| Challenge: | In-context learning is a new learning paradigm where a language model observes a few examples and directly outputs the test input’s prediction. |
| Approach: | They propose a method to find “support examples” for in-context learning by filtering a training dataset and a progressive filtering process to filter out uninformative examples. |
| Outcome: | The proposed method outperforms baselines and shows that each component contributes critically to the improvements. |
Similar Papers
Active Example Selection for In-Context Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning performance is unstable across samples of examples, suggesting the idiosyncrasies of how language models acquire information. |
| Approach: | They propose a reinforcement learning algorithm for identifying generalizable policies to select demonstration examples and propose 'in-context learning' performance can be highly unstable across samples of examples, suggesting the idiosyncrasies of how language models acquire information. |
| Outcome: | The proposed model can perform tasks with examples with a 5.8% improvement on GPT-2 and GPT-3, but the improvement diminishes on larger models, suggesting emerging capabilities of large language models. |
Effective In-Context Example Selection through Data Compression (2024.findings-acl)
Copied to clipboard
| Challenge: | In-context learning has been validated in large language models, but the mechanism and selection strategy for in-cont example selection lacks systematic and in-depth research. |
| Approach: | They propose a data compression approach to select in-context examples using large language models. |
| Outcome: | The proposed method shows a significant improvement of 5.90% across five real-world datasets using four language models. |
Learning to Retrieve In-Context Examples for Large Language Models (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing approaches to improve in-context learning performance are highly sensitive to the quality of the incontext examples provided. |
| Approach: | They propose a framework to iteratively train dense retrievers that can identify high-quality in-context examples for large language models. |
| Outcome: | The proposed model improves performance by retrieving examples with similar patterns, and the gains are consistent across LLMs of varying sizes. |
Coverage-based Example Selection for In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a training-free paradigm of fewshot inference that can generalize to novel tasks by conditioning on a few task examples. |
| Approach: | They show that BERTScore-Recall (BSR) selects better examples that demonstrate more of the salient aspects of the test input. |
| Outcome: | The proposed model outperforms methods that leverage task or LLM-specific training on compositional tasks. |
Understanding In-Context Learning via Supportive Pretraining Data (2023.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a form of learning that provides a handful of examples at inference time, but it is not well understood why it emerges as the model has never been specifically trained on such demonstrations. |
| Approach: | They adapt an iterative, gradient-based approach to find a small subset of pretraining data that supports ICL and compare it with random subsets of pretrain data. |
| Outcome: | The proposed method improves the model's ICL ability by 18% if it is continued on a small subset of pretraining data. |
Structured Semantic Information Helps Retrieve Better Examples for In-Context Learning Applied to Few-Shot Relation Extraction (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to relation extraction require many training examples per relation, resulting in low results. |
| Approach: | They propose a strategy where new examples are selected based on their similarity to the provided 1-shot example. |
| Outcome: | The proposed strategy outperforms other methods on FS-TACRED and FS - FewRel subsets and achieves state-of-the-art performance on both datasets. |
In-context Examples Selection for Machine Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Large-scale generative models can perform a wide range of NLP tasks using in-context learning. |
| Approach: | They aim to understand the properties of good in-context examples for machine translation in both in-domain and out-of-domain settings. |
| Outcome: | The proposed model outperforms a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets. |
Revisiting In-Context Learning with Long Context Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) is a technique by which language models make predictions based on examples provided in their input context. |
| Approach: | They revisited previous studies using in-context learning techniques . they found that using a data augmentation approach, they significantly improved ICL performance . |
| Outcome: | The proposed approach significantly improves ICL performance on 18 datasets spanning 4 tasks . the proposed approach does not improve performance over a simple random sample selection method . |
Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on in-context learning mechanisms are not consistent . current research identifies two main approaches to explain the ICL mechanism . |
| Approach: | They propose a framework for evaluating in-context learning mechanisms by focusing on regression tasks. |
| Outcome: | The proposed framework can solve regression problems and then measure the extent to which the LLM retrieves its internal knowledge versus learning from in-context examples. |
Meta-learning via Language Model In-context Tuning (2022.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models have reduced "task learning and prediction" to a simple sequence prediction problem. |
| Approach: | They propose a meta-learning method that recasts task adaptation and prediction as a sequence prediction problem. |
| Outcome: | The proposed method outperforms MAML on two classification tasks and improves on binaryClfs. |