Investigating the Zone of Proximal Development of Language Models for In-Context Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | In-context learning is a dynamic and progressive process where learners integrate new information into their knowledge base through interactions with the environment. |
| Approach: | They propose a learning analytics framework to analyze the in-context learning behavior of large language models (LLMs) through the lens of the Zone of Proximal Development (ZPD), an established theory in educational psychology. |
| Outcome: | The proposed framework improves inference and fine-tuning scenarios by selectively applying it to queries that are most likely to benefit from demonstrations. |
Similar Papers
Fine-tuning vs. In-context Learning in Large Language Models: A Formal Language Learning Perspective (2026.acl-long)
Copied to clipboard
Bishwamittra Ghosh, Soumi Das, Till Speicher, Qinyuan Wu, Mohammad Aflah Khan, Deepak Garg, Krishna P. Gummadi, Evimaria Terzi
| Challenge: | Prior studies comparing FT and ICL have yielded mixed and inconclusive results due to inconsistent experimental setups. |
| Approach: | They propose a formal language learning task with precise language boundaries, controlled string sampling, and no data contamination to enable a rigorous comparison. |
| Outcome: | The proposed task offers precise language boundaries, controlled string sampling, and no data contamination. |
A Survey on In-context Learning (2024.emnlp-main)
Copied to clipboard
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui
| Challenge: | In-context learning (ICL) is a new paradigm for natural language processing . large language models (LLMs) demonstrate the ability to learn from a few examples . |
| Approach: | They propose to explore ICL to evaluate and extrapolate the ability of large language models. |
| Outcome: | The proposed methods can be used to evaluate and extrapolate the ability of large language models. |
An Empirical Study of In-context Learning in LLMs for Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies focus on optimizing translation quality, with limited attention to understanding specific aspects of ICL that influence the said quality. |
| Approach: | They conduct the first of its kind, exhaustive study of in-context learning for machine translation (MT) they establish that ICL is primarily example-driven and not instruction-driven . |
| Outcome: | The proposed model is based on examples and not instruction-driven learning. |
What Do Language Models Learn in Context? The Structured Task Hypothesis. (2024.acl-long)
Copied to clipboard
| Challenge: | Pre-trained large language models have exhibited an impressive ability to learn in context across various domains, e.g., code generation, education, medicine and even medicine. |
| Approach: | They taxonomize existing candidate theories into three competing hypotheses that explain LLMs’ ability to learn in context. |
| Outcome: | The proposed model can learn a task from in-context examples presented in a demonstration and generalize it to the prompt. |
The Mystery of In-Context Learning: A Comprehensive Survey on Interpretation and Analysis (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a capability that enables large language models to excel in proficiency through demonstration examples. |
| Approach: | They present a survey on the interpretation and analysis of in-context learning . they focus on theoretical and empirical perspectives on the concept . |
| Outcome: | The proposed model can perform tasks with minimal examples without re-training and has demonstrated proficiency across various tasks with a minimal set of task-oriented examples. |
Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on in-context learning mechanisms are not consistent . current research identifies two main approaches to explain the ICL mechanism . |
| Approach: | They propose a framework for evaluating in-context learning mechanisms by focusing on regression tasks. |
| Outcome: | The proposed framework can solve regression problems and then measure the extent to which the LLM retrieves its internal knowledge versus learning from in-context examples. |
In-Context Learning Creates Task Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a powerful new learning paradigm for Large Language Models (LLMs). |
| Approach: | They propose to use a model with a prompt and a query to learn a mapping based on two examples to produce the output. |
| Outcome: | The proposed model can learn functions from a simple structure based on a training set and a single task vector calculated from the training set. |
Beyond In-Context Learning: Aligning Long-form Generation of Large Language Models via Task-Inherent Attribute Guidelines (2025.findings-acl)
Copied to clipboard
Do Xuan Long, Duong Ngoc Yen, Do Xuan Trong, Anh Tuan Luu, Kenji Kawaguchi, Shafiq Joty, Min-Yen Kan, Nancy F. Chen
| Challenge: | In-context learning is an important but not fully understood ability of pre-trained large language models. |
| Approach: | They propose a tool that generates two streams of guidelines capturing task language and format distributions and prompts them to define them by prompting. |
| Outcome: | The proposed model improves both strong open- and closed-source LLMs by over 5% in both zero- and few-shot settings. |
ICLEval: Evaluating In-Context Learning Ability of Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Existing evaluation frameworks focus on language abilities and knowledge, often overlooking the assessment of ICL ability. |
| Approach: | They propose to evaluate the ICL ability of Large Language Models (LLMs) using the ICLEval benchmark. |
| Outcome: | The proposed benchmark demonstrates that ICL ability is universally present in different LLMs and model size is not the sole determinant of ICL efficacy. |
Tutor-ICL: Guiding Large Language Models for Improved In-Context Learning Performance (2024.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a dominant paradigm in natural language processing. |
| Approach: | They propose a prompting method for classification tasks using exemplar answers in a *comparative format' they also propose introducing a test instance before the exemplars to improve performance . |
| Outcome: | The proposed method achieves up to 13.76% increase in accuracy on classification tasks across decoder-only and encoder-decoder LLMs. |