| Challenge: | Existing methods to improve ICL performance are limited by the length of the input context. |
| Approach: | They propose a method that utilizes all demonstration examples without exceeding the manageable context length. |
| Outcome: | The proposed method can be scaled up to integrate with existing methods. |
Similar Papers
Focused Large Language Models are Stable Many-Shot Learners (2024.emnlp-main)
Copied to clipboard
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li
| Challenge: | In-Context Learning (ICL) enables large language models to achieve rapid task adaptation by learning from demonstrations. |
| Approach: | They propose a training-free method that disperses model attention from the query . they propose 'focus' search strategy that uses model perplexity to ensure sufficient attention . |
| Outcome: | The proposed method achieves an average performance improvement of 5.2% over vanilla ICL and scales well with many-shot demonstrations. |
In-Context Learning with Iterative Demonstration Selection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing literature has highlighted the importance of selecting examples that are diverse or semantically similar to the test sample . Existing studies have shown that the optimal selection dimension, i.e., diversity or similarity, is task-specific. |
| Approach: | They propose to use zero-shot chain-of-thought reasoning to iteratively select examples that are diverse but still strongly correlated with the test sample as ICL demonstrations. |
| Outcome: | The proposed method outperforms existing demonstration selection methods on reasoning, question answering, and topic classification tasks. |
MetaICL: Learning to Learn In Context (2022.naacl-main)
Copied to clipboard
| Challenge: | Large language models can do in-context learning by conditioning on a few training examples with no parameter updates or task-specific templates. |
| Approach: | They propose a meta-training framework where a pretrained language model is tuned to do in-context learning on a large set of training tasks. |
| Outcome: | The proposed framework outperforms baseline models on 142 NLP datasets and a range of target tasks with domain shifts. |
On Many-Shot In-Context Learning for Long-Context Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks primarily evaluate long-context language models' retrieval capabilities. |
| Approach: | They propose a benchmark to evaluate long-context language models' retrieval capabilities by using MANYICLBENCH. |
| Outcome: | The proposed model performs better with additional demonstrations than translation and reasoning tasks. |
Unraveling the Mechanics of Learning-Based Demonstration Selection for In-Context Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Recent learning-based demonstration selection methods have proven beneficial to in-context learning (ICL) by choosing more useful exemplars. |
| Approach: | They propose two methods to capture task-agnostic similarities between input and output of LLMs. |
| Outcome: | The proposed methods integrate task-agnostic similarities of different levels between input and output of exemplars and test cases to eliminate costly data collection. |
Learning to Select In-Context Demonstration Preferred by Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | In-context learning (ICL) enables large language models to perform tasks with only a few examples as demonstrations. |
| Approach: | They propose a generative preference learning framework that leverages LLM feedback to directly optimize demonstration selection for ICL. |
| Outcome: | Experiments on 19 datasets across 11 task categories show that GenICL achieves superior performance than existing methods in selecting the most effective demonstrations. |
Self-ICL: Zero-Shot In-Context Learning with Self-Generated Demonstrations (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown striking ability to adapt to target tasks with a few input-output demonstrations. |
| Approach: | They propose a framework which bootstraps LMs’ intrinsic capabilities to perform zero-shot ICL. |
| Outcome: | The proposed framework outperforms baselines on 23 BIG-Bench Hard tasks on average accuracy and head-to-head comparison. |
Se2: Sequential Example Selection for In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Prior work has explored the selection of examples for in-context learning, neglecting the internal relationships between examples and exist an inconsistency between training and inference. |
| Approach: | They propose a sequential-aware method that leverages the LLM’s feedback on varying context, aiding in capturing inter-relationships and sequential information among examples. |
| Outcome: | Experiments on 23 NLP tasks show that Se2 surpasses baselines and achieves 42% relative improvement over random selection. |
OpenICL: An Open-Source Framework for In-context Learning (2023.acl-demo)
Copied to clipboard
| Challenge: | In-context Learning (ICL) is a new paradigm for large language model evaluation. |
| Approach: | They propose an open-source toolkit for ICL and LLM evaluation. |
| Outcome: | The proposed framework is highly flexible and flexible and can be easily combined with other tools to suit users' needs. |
Revisiting In-Context Learning with Long Context Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) is a technique by which language models make predictions based on examples provided in their input context. |
| Approach: | They revisited previous studies using in-context learning techniques . they found that using a data augmentation approach, they significantly improved ICL performance . |
| Outcome: | The proposed approach significantly improves ICL performance on 18 datasets spanning 4 tasks . the proposed approach does not improve performance over a simple random sample selection method . |