IN-CONTEXT

DIALIGHT: Lightweight Multilingual Development and Evaluation of Task-Oriented Dialogue Systems with Large Language Models (2024.naacl-demo)

Challenge:	General-purpose Language Models have changed the world of Natural Language Processing, if not the world itself.
Approach:	This tutorial will lay the foundations and explain the basics of evaluation and compare traditional methods to newly developed methods.
Outcome:	The tutorial assumes little familiarity with metrics, datasets, prompts and benchmarks . it will compare traditional methods to newly developed methods .

Copied to clipboard

Songbo Hu, Xiaobin Wang, Moy Yuan, Anna Korhonen, Ivan Vulić

Natural Language Interfaces

Challenge:	DIALIGHT is a toolkit for developing and evaluating multilingual Task-Oriented Dialogue systems.
Approach:	They propose a toolkit for developing and evaluating multilingual Task-Oriented Dialogue systems which facilitates systematic evaluations and comparisons between ToD systems using pretrained language models and those utilising the zero-shot and in-context learning capabilities of Large Language Models.
Outcome:	The toolkit enables systematic evaluations between ToD systems using pretrained language models and those utilising the zero-shot and in-context learning capabilities of Large Language Models (LLMs).

Scaling Context, Not Parameters: Training a Compact 7B Language Model for Efficient Long-Context Processing (2025.acl-industry)

Copied to clipboard

Chen Wu, Yin Song

Gradient-Attention Guided Dual-Masking Synergetic Framework for Robust Text-based Person Retrieval (2025.emnlp-main)

Challenge:	a new language model that supports 512K context lengths addresses practical limitations in long-context training . a competitive 35% score on 512k-token BABILong tasks without RAG or task-specific tuning is achieved .
Approach:	They present a language model that supports 512K-token context length . they evaluated its long-context learning performance on three benchmarks .
Outcome:	The model outperforms open-source models on three long-context benchmarks . it achieves a competitive 35% score on 512K-token BABILong tasks without RAG or fine-tuning .

Copied to clipboard

Tianlu Zheng, Yifan Zhang, Xiang An, Ziyong Feng, Kaicheng Yang, Qichuan Ding

Pelican Soup Framework: A Theoretical Framework for Language Model Capabilities (2026.findings-eacl)

Challenge:	a large-scale visionlanguage pre-training framework is limited by the scarcity of large-sized annotated vision-language data . noise-resistant data construction pipeline is needed to filter and caption web-sourced images . noisy text tokens can be a problem for fine-grained representation learning .
Approach:	They develop a noise-resistant data construction pipeline that leverages in-context learning capabilities of MLLMs to automatically filter and caption web-sourced images.
Outcome:	The proposed framework improves cross-modal alignment by masking noisy textual tokens based on the gradient-attention similarity score.

Copied to clipboard

Ting-Rui Chiang, Dani Yogatama

LLMs Are Few-Shot In-Context Low-Resource Language Learners (2024.naacl-long)

Challenge:	Existing theoretical frameworks for large language models (LLMs) do not explain how pretraining leads to in-context learning.
Approach:	They propose a theoretical framework that allows LLMs to generalize to unseen instructions and perform in-context learning even when verbalizers are irrelevant to the task.
Outcome:	The proposed framework can be used to analyze LLMs' ability to perform in-context learning . it can be applied to linguistic, psychology, and philosophy tasks .

Copied to clipboard

Samuel Cahyawijaya, Holy Lovenia, Pascale Fung

On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choices (2024.emnlp-main)

Challenge:	In-context learning (ICL) empowers large language models to perform diverse tasks in underrepresented languages using only short in-contrast information.
Approach:	They extensively assess the effectiveness of in-context learning with LLMs in low-resource languages . they also identify the shortcomings of in context label alignment .
Outcome:	The proposed approach improves understanding quality of low-resource languages by closing the language gap in the target language.

Copied to clipboard

Branislav Pecher, Ivan Srba, Maria Bielikova

Multilingual Retrieval-Augmented Generation for Knowledge-Intensive Question Answering Task (2026.findings-eacl)

Challenge:	Using limited labelled data, learning with limited labels is sensitive to the effects of uncontrolled randomness.
Approach:	They propose to investigate the effects of individual randomness factors while taking the interactions between them into consideration.
Outcome:	The proposed method mitigates the effects of other factors while observing how the performance varies across multiple runs.

Copied to clipboard

Leonardo Ranaldi, Barry Haddow, Alexandra Birch

Multilinguality

Challenge:	Existing studies focus on English as the data language for RAG, resulting in limited coverage of multilingual RAG.
Approach:	They propose a method that translates retrieved documents into a common language before generating the response.
Outcome:	The proposed approach improves efficiency on knowledge-intensive tasks but introduces inconsistencies due to cross-lingual variations in the retrieved content.

The AI Committee: A Multi-Agent Framework for Automated Validation and Remediation of Web-Sourced Data (2026.eacl-demo)

Copied to clipboard

Sunith Vallabhaneni, Thomas Berkane, Maimuna S. Majumder

Challenge:	largelanguage models (LLMs)-powered web agents can be useful for research in areas such as social science, public health, and economics.
Approach:	They propose a model-agnostic multi-agent system that auto-mates the process of validating and remediatingweb-sourced datasets.
Outcome:	The proposed system outperforms baseline approaches and achieves datacompleteness and precision up to 73.3%.

SPICA: Retrieving Scenarios for Pluralistic In-Context Alignment (2025.findings-acl)

Copied to clipboard

Quan Ze Chen, Kevin Feng, Chan Young Park, Amy X Zhang

Unsupervised Contrast-Consistent Ranking with Language Models (2024.eacl-long)

Challenge:	In-context learning only considers similarity when drawing few-shot examples and not cross-group differences in values.
Approach:	They propose a framework that accounts for group-level differences during in-context example retrieval by using scenario banks, group-informed retrieval metrics, and in-constraint alignment prompts.
Outcome:	The proposed framework improves on an alignment task with groups seeing up to a +0.16 point improvement on a 5 point scale.

Copied to clipboard

Niklas Stoehr, Pengxiang Cheng, Jing Wang, Daniel Preotiuc-Pietro, Rajarshi Bhowmik

Symbol tuning improves in-context learning in language models (2023.emnlp-main)

Challenge:	Language models contain ranking-based knowledge and are powerful solvers of in-context ranking tasks.
Approach:	They propose to use a model to elicit language models' ranking knowledge without supervision by using a pairwise, pointwise and listwise prompting method.
Outcome:	The proposed method is inspired by an unsupervised probing method called Contrast-Consistent Search (CCS).

Copied to clipboard

Jerry Wei, Le Hou, Andrew Lampinen, Xiangning Chen, Da Huang, Yi Tay, Xinyun Chen, Yifeng Lu, Denny Zhou, Tengyu Ma, Quoc Le

Language Models “Grok” to Copy (2025.naacl-short)

Challenge:	Language models are sensitive to the way that prompts are given, indicating that they are not reasoning in a robust manner.
Approach:	They propose to fine tune language models on in-context input-label pairs where natural language labels are replaced with arbitrary symbols.
Outcome:	The proposed model is much stronger at reasoning tasks and more robust to underspecified prompts than the standard model.

Copied to clipboard

Ang Lv, Ruobing Xie, Xingwu Sun, Zhanhui Kang, Rui Yan

Towards the Machine Translation of Scientific Neologisms (2025.coling-main)

Challenge:	We examine the pre-training dynamics of language models, focusing on their ability to copy text from preceding context.
Approach:	They propose that Transformer-based language models develop copying abilities similarly to grokking . they argue that the connection between groking and context copying can improve in-context performance.
Outcome:	The proposed model development is similar to grokking, but the speed is independent of tokens trained.

Copied to clipboard

Paul Lerner, François Yvon

In-Context Example Selection via Similarity Search Improves Low-Resource Machine Translation (2025.findings-naacl)

Challenge:	Scientific research continually discovers and invents new concepts, which are then referred to by new terms, neologisms, or nenonyms.
Approach:	They propose to leverage term definitions to translate neologisms with Large Language Models . they find that LLMs generate terms from co-hyponyms and terms sharing the same derivation paradigm .
Outcome:	The proposed model can generate terms from co-hyponyms and terms sharing the same derivation paradigm.

Copied to clipboard

Armel Randy Zebaze, Benoît Sagot, Rachel Bawden

Text Generation

Challenge:	Existing studies have shown that in-context examples for machine translation are beneficial for high-resource languages.
Approach:	They propose to use in-context examples for machine translation (MT) they argue that similarity-based selection can improve MT .
Outcome:	The proposed approach improves machine translation (MT) and low-resource languages.

In-Context Demonstration Selection with Cross Entropy Difference (2023.findings-emnlp)

Copied to clipboard

Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu

What Makes Chain-of-Thought Prompting Effective? A Counterfactual Study (2023.findings-emnlp)

Challenge:	Large language models (LLMs) can use in-context demonstrations to improve performance on zero-shot tasks.
Approach:	They propose a cross-entropy difference method for selecting in-context demonstrations that uses parameter efficient finetuning to train small models on training data.
Outcome:	The proposed method outperforms baseline selection methods on a mix-domain dataset and shows that the effectiveness of in-context demonstrations negatively correlates with the perplexity of the test example.

Copied to clipboard

Aman Madaan, Katherine Hermann, Amir Yazdanbakhsh

Value Profiles for Encoding Human Variation (2025.emnlp-main)

Challenge:	Using a few-shot prompt, we examine the effects of symbols and patterns on in-context learning in large language models.
Approach:	They employ a counterfactual prompting approach by manipulating examples and testing the consequences on model behavior.
Outcome:	The proposed approach allows us to understand the relative contributions of symbols and patterns on in-context learning.

Copied to clipboard

Taylor Sorensen, Pushkar Mishra, Roma Patel, Michael Henry Tessler, Michiel A. Bakker, Georgina Evans, Iason Gabriel, Noah Goodman, Verena Rieser

Explainability & Interpretability in NLP

Challenge:	Using value profiles and a steerable decoder model to estimate ratings is crucial for personalization, pluralistic model alignment, and computational social science.
Approach:	They propose to represent individuals using value profiles and a steerable decoder model to estimate ratings conditioned on a value profile or other rater information.
Outcome:	The proposed model interpretably changes ratings according to semantic profile differences and is well-calibrated.

Selective In-Context Data Augmentation for Intent Detection using Pointwise V-Information (2023.eacl-main)

Copied to clipboard

Yen-Ting Lin, Alexandros Papangelis, Seokhwan Kim, Sungjin Lee, Devamanyu Hazarika, Mahdi Namazifar, Di Jin, Yang Liu, Dilek Hakkani-Tur

Crafting In-context Examples according to LMs’ Parametric Knowledge (2024.findings-naacl)

Challenge:	Intent detection is a fundamental element in task-oriented dialogue systems, usually occurring within the Natural Language Understanding component.
Approach:	They propose an in-context data augmentation approach that fine-tunes a pre-trained language model and synthesizes new datapoints that correspond to given intents.
Outcome:	The proposed method produces training data that achieves state-of-the-art on three challenging intent detection datasets and performs on par with the state- of-the art in full-shot settings.

Copied to clipboard

Yoonsang Lee, Pranav Atreya, Xi Ye, Eunsol Choi

Question Answering

Challenge:	In-context examples can improve the performance of knowledge-rich tasks such as question answering by triggering a language model to surface information stored in its parametric knowledge.
Approach:	They propose to construct in-context example sets based on model's parametric knowledge by prompting models with 'unknown' examples.
Outcome:	The proposed model can perform better on in-context examples in three multi-answer question answering datasets, and prompting with ‘unknown’ examples decreases the performance.

Are Hard Examples also Harder to Explain? A Study with Human and Model-Generated Explanations (2022.emnlp-main)

Copied to clipboard

Swarnadeep Saha, Peter Hase, Nazneen Rajani, Mohit Bansal

Explainability & Interpretability in NLP

Challenge:	Recent work on explainable NLP has shown that few-shot prompting can enable large pre-trained language models (LLMs) to generate grammatical and factual explanations for data labels.
Approach:	They propose to use few-shot prompting to generate grammatical and factual explanations for data labels by varying the hardness of the test samples and in-context samples to investigate the link between explainability and sample hardness.
Outcome:	The proposed model can generate grammatical and factual explanations for data labels with few-shot prompting.

Generative Calibration for In-context Learning (2023.findings-emnlp)

Copied to clipboard

Zhongtao Jiang, Yuanzhe Zhang, Cao Liu, Jun Zhao, Kang Liu

Enhancing In-Context Learning via Implicit Demonstration Augmentation (2024.acl-long)

Challenge:	In-context learning is one of the most exciting features of large language models . performance is sensitive to various configurations of the prompt, such as the choice or order of the training examples.
Approach:	They propose to calibrate the in-context predictive distribution by adjusting the label marginal . they find that the proposed method outperforms the ICL and state-of-the-art calibration methods .
Outcome:	The proposed method outperforms state-of-the-art methods by 27% absolute in macro-F1.

Copied to clipboard

Xiaoling Zhou, Wei Ye, Yidong Wang, Chaoya Jiang, Zhemg Lee, Rui Xie, Shikun Zhang

In-Context Example Ordering Guided by Label Distributions (2024.findings-naacl)

Challenge:	In-context learning (ICL) is a new paradigm for pre-trained language models that can make predictions for unseen inputs without updating parameters.
Approach:	They propose a method that enables a model to augmented copies of a demonstration by leveraging their deep feature distribution and a logit calibration mechanism.
Outcome:	The proposed method significantly improves the average and worst-case accuracy across diverse PLMs and tasks.

Copied to clipboard

Zhichao Xu, Daniel Cohen, Bei Wang, Vivek Srikumar

mOSCAR: A Large-scale Multilingual and Multimodal Document-level Corpus (2025.findings-acl)

Challenge:	Prior work has shown that ICL is sensitive to different natural language instructions and different orderings of in-context examples.
Approach:	They propose two principles for in-context example ordering guided by model’s probability predictions.
Outcome:	The proposed model outperforms baseline models on 13 text classification datasets and nine autoregressive LLMs with 700M to 13B parameters.

Copied to clipboard

Matthieu Futeral, Armel Randy Zebaze, Pedro Ortiz Suarez, Julien Abadji, Rémi Lacroix, Cordelia Schmid, Rachel Bawden, Benoît Sagot

How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning (2024.emnlp-main)

Challenge:	Existing studies show that multimodal large language models can learn from text-image data.
Approach:	They propose to train multimodal large language models on large amounts of text-image data . they also show a boost in few-shot learning performance across various multilingual tasks .
Outcome:	The proposed dataset is not public and is only in English . it is the first large-scale multilingual and multimodal document corpus crawled from the web.

Copied to clipboard

Zeping Yu, Sophia Ananiadou

Exploring Automated Distractor Generation for Math Multiple-choice Questions via Large Language Models (2024.findings-naacl)

Challenge:	In-context learning (ICL) is an emergent ability of large language models.
Approach:	They propose to use in-context learning to predict sentences with semantically-unrelated labels on 1% heads to investigate the mechanism.
Outcome:	The proposed methods reduce the majority label bias and recency bias by 22% and 17%, respectively.

Copied to clipboard

Wanyong Feng, Jaewook Lee, Hunter McNichols, Alexander Scarlatos, Digory Smith, Simon Woodhead, Nancy Ornelas, Andrew Lan

In-context Learning for Few-shot Multimodal Named Entity Recognition (2023.findings-emnlp)

Challenge:	Multiple-choice questions (MCQs) are easy to administer and grade . but crafting high-quality distractors remains labor-intensive and limited scalability .
Approach:	They propose to automate the generation of distractors in math MCQs by using large language models to generate distractors.
Outcome:	The proposed methods can generate valid distractors, but they are less adept at anticipating common errors or misconceptions among real students.

Copied to clipboard

Chenran Cai, Qianlong Wang, Bin Liang, Bing Qin, Min Yang, Kam-Fai Wong, Ruifeng Xu

Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts (2022.findings-emnlp)

Challenge:	Existing methods for named entity recognition are time-consuming and laborintensive.
Approach:	They propose a few-shot multimodal named entity recognition task that uses few examples to locate and identify named entities for a text-image pair.
Outcome:	The proposed framework outperforms baselines under several few-shot settings.

Copied to clipboard

Nghia T. Le, Fan Bai, Alan Ritter

INFORM : Information eNtropy based multi-step reasoning FOR large language Models (2023.emnlp-main)

Challenge:	In-context learning has emerged as a promising approach to resolve anaphora, but there are challenges in applying it to scientific protocols.
Approach:	They propose a method which combines predictions of hundreds of in-context experts and combines them to yield a 30% increase in F1 over a competitive prompt retrieval baseline.
Outcome:	The proposed method yields 30% increase in F1 score over a competitive prompt retrieval baseline.

Copied to clipboard

Chuyue Zhou, Wangjie You, Juntao Li, Jing Ye, Kehai Chen, Min Zhang

Better Zero-Shot Reasoning with Self-Adaptive Prompting (2023.findings-acl)

Challenge:	Large language models (LLMs) have demonstrated exceptional performance with dedicated Chain-of-Thought (CoT) prompts.
Approach:	They propose a new method by introducing information entropy as a criteria on for CoT prompt selection.
Outcome:	The proposed model outperforms existing models on seven reasoning benchmarks using two language models.

Copied to clipboard

Xingchen Wan, Ruoxi Sun, Hanjun Dai, Sercan Arik, Tomas Pfister

XAMPLER: Learning to Retrieve Cross-Lingual In-Context Examples (2025.findings-naacl)

Challenge:	Modern large language models (LLMs) have demonstrated impressive capabilities at sophisticated tasks, often through step-by-step reasoning similar to humans.
Approach:	They propose a new method that uses a set of examples from the LLM zero-shot outputs to improve performance.
Outcome:	The proposed method improves performance up to 15% compared to baselines and matches or exceeds few-shot baselines at a range of reasoning tasks.

Copied to clipboard

Peiqin Lin, Andre Martins, Hinrich Schuetze

Why Can GPT Learn In-Context? Language Models Secretly Perform Gradient Descent as Meta-Optimizers (2023.findings-acl)

Challenge:	XAMPLER: Cross-Lingual Example Retrieval is a cross-lingual example retrieval method . large language models (LLMs) have emerged as effective in-context learning methods .
Approach:	They propose a method to train a multilingual model with annotated English examples . they use annotized English data to train the model and use it to train other languages .
Outcome:	XAMPLER: Cross-Lingual Example Retrieval improves in-context learning in English . it trains a retriever based on a multilingual small language model using annotated English examples .

Copied to clipboard

Damai Dai, Yutao Sun, Li Dong, Yaru Hao, Shuming Ma, Zhifang Sui, Furu Wei

Memo-SQL: Structured Decomposition and Experience-Driven Self-Correction for Training-Free NL2SQL (2026.findings-acl)

Challenge:	Large pretrained language models have shown surprising in-context learning ability . despite the great success in performance, its working mechanism remains unclear .
Approach:	They explain language models as meta-optimizers and understand in-context learning as implicit finetuning . they find that Transformer attention has a dual form of gradient descent .
Outcome:	The proposed model can predict labels for unseen inputs without parameter updates . the proposed model outperforms smaller models with a single parameter update .

Copied to clipboard

Yang Zerui, Weichuan Wang, Yanwei Xu, Linqi Song, Yudai Matsuda, Wei Han, Bo Bai

Programming Languages in NLP

Challenge:	Existing NL2SQL systems rely on in-context learning with only correct examples . current test-time scaling methods often decompose questions arbitrarily, resulting in poor performance .
Approach:	They propose a structured decomposition and experience-aware self-correction framework for NL2SQL . they build a dynamic memory of successful queries and historical error–fix pairs .
Outcome:	The proposed framework achieves 68.5% execution accuracy on BIRD, setting new state of the art among open, zero-fine-tuning methods.

Pre-Training to Learn in Context (2023.acl-long)

Copied to clipboard

Yuxian Gu, Li Dong, Furu Wei, Minlie Huang

What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers (2021.emnlp-main)

Challenge:	Pre-trained language models are not explicitly trained to learn in context.
Approach:	They propose a framework to enhance in-context learning by pre-training language models on a large collection of "intrinsic tasks" they evaluate the in-constitution learning performance of the model trained with PICL on seven widely-used text classification datasets and the Super-NaturalInstrctions benchmark .
Outcome:	The proposed framework outperforms larger language models with nearly 4x parameters on seven widely-used datasets and the Super-NaturalInstrctions benchmark.

Copied to clipboard

Boseop Kim, HyoungSeok Kim, Sang-Woo Lee, Gichang Lee, Donghyun Kwak, Jeon Dong Hyeon, Sunghyun Park, Sungju Kim, Seonhoon Kim, Dongpil Seo, Heungsub Lee, Minyoung Jeong, Sungjae Lee, Minsub Kim, Suk Hyun Ko, Seokhun Kim, Taeyong Park, Jinuk Kim, Soyoung Kang, Na-Hyeon Ryu, Kang Min Yoo, Minsuk Chang, Soobin Suh, Sookyo In, Jinseong Park, Kyungduk Kim, Hiun Kim, Jisu Jeong, Yong Goo Yeo, Donghoon Ham, Dongju Park, Min Young Lee, Jaewook Kang, Inho Kang, Jung-Woo Ha, Woomyoung Park, Nako Sung

OLMES: A Standard for Language Model Evaluations (2025.findings-naacl)

Challenge:	GPT-3 has been used to train large-scale language models on hundreds of billion scale data.
Approach:	They propose a Korean variant of GPT-3 that uses Korean tokens to train in-context models.
Outcome:	The proposed method shows state-of-the-art zero-shot and few-shot learning on downstream tasks in Korean.

Copied to clipboard

Yuling Gu, Oyvind Tafjord, Bailey Kuehl, Dany Haddad, Jesse Dodge, Hannaneh Hajishirzi

Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning (2023.findings-acl)

Challenge:	Existing models claim to perform better on tasks measuring model capabilities, but there is no standard setup for reproducible evaluations.
Approach:	They propose a document that is documented and practical for reproducible LLM evaluations and includes recommendations from existing literature and new experiments.
Outcome:	The proposed standard identifies and reviews the varying factors in evaluation practices adopted by the community, such as prompt formatting, choice of in-context examples, probability normalizations, and task formulation.

Copied to clipboard

Ruixiang Tang, Dehan Kong, Longtao Huang, Hui Xue

Information Extraction from Legal Wills: How Well Does GPT-4 Do? (2023.findings-emnlp)

Challenge:	Large language models (LLMs) have shown great potential for in-context learning, but their robustness and performance on downstream tasks remains limited.
Approach:	They propose to examine the reliance of LLMs on shortcuts or spurious correlations within prompts for downstream tasks and find larger models are more likely to utilize shortcuts in prompts during inference.
Outcome:	The proposed model is “lazy learner” and more likely to use shortcuts in prompts during inference.

Copied to clipboard

Alice Kwak, Cheonkam Jeong, Gaetano Forte, Derek Bambauer, Clayton Morrison, Mihai Surdeanu

BRIEF: Bridging Retrieval and Inference for Multi-hop Reasoning via Compression (2025.findings-naacl)

Challenge:	Using information extraction from legal wills is an important application of artificial intelligence (AI)
Approach:	They propose a manually annotated dataset for Information Extraction (IE) from legal wills . they also use it to evaluate the performance of large language models (LLMs)
Outcome:	The proposed dataset can be used to evaluate large language models on IE from legal wills . it shows that the model performs reasonably well, but inconsistent outputs and overgeneralization are observed .

Copied to clipboard

Yuankai Li, Jia-Chen Gu, Di Wu, Kai-Wei Chang, Nanyun Peng

How Does the Textual Information Affect the Retrieval of Multimodal In-Context Learning? (2024.emnlp-main)

Challenge:	Existing approaches to augment language models with external knowledge but they are limited by static nature of pre-training data.
Approach:	They propose a lightweight approach that compresses retrieved documents into highly dense textual summaries to integrate into in-context RAG.
Outcome:	The proposed approach reduces latency and costs while achieving high performance in open-domain questions.

Copied to clipboard

Yang Luo, Zangwei Zheng, Zirui Zhu, Yang You

Learning Geometry-Aware Representations for New Intent Discovery (2024.acl-long)

Challenge:	MLLMs have significant capabilities for multimodal in-context learning, but their effectiveness hinges on the appropriate selection of in-constext examples.
Approach:	They propose a supervised MLLM prompt retriever that leverages a trained retriever based on MLML's confidence to select examples, which enhances multimodal in-context learning efficiency.
Outcome:	The proposed method is validated through extensive testing across three different tasks and demonstrates its effectiveness.

Copied to clipboard

Kai Tang, Junbo Zhao, Xiao Ding, Runze Wu, Lei Feng, Gang Chen, Haobo Wang

Information Extraction & Text Mining

Challenge:	Existing methods for intent classification fail to distinguish new intents due to intertwined centers . a novel framework that learns geometry-aware representations to maximally separate all intents is proposed .
Approach:	They propose a new intent discovery framework that learns geometry-aware representations to maximally separate all intents.
Outcome:	The proposed framework achieves a new state-of-the-art performance on three benchmarking datasets.

Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Again (2022.findings-emnlp)

Copied to clipboard

Bernal Jimenez Gutierrez, Nikolas McNeal, Clayton Washington, You Chen, Lang Li, Huan Sun, Yu Su

Representative Demonstration Selection for In-Context Learning with Two-Stage Determinantal Point Process (2023.emnlp-main)

Challenge:	Large pre-trained language models (PLMs) such as GPT-3 have shown strong in-context learning capabilities, which are appealing for domains such as biomedicine that feature high and diverse demands of language technologies but also high data annotation costs.
Approach:	They propose to compare the few-shot performance of GPT-3 in-context learning with fine-tuning smaller (i.e., BERT-sized) PLMs on two representative biomedical information extraction tasks: named entity recognition and relation extraction.
Outcome:	The proposed model underperforms on two representative biomedical information extraction tasks.

Copied to clipboard

Zhao Yang, Yuanzhe Zhang, Dianbo Sui, Cao Liu, Jun Zhao, Kang Liu

impact of sample selection on in-context learning for entity extraction from scientific writing (2023.findings-emnlp)

Challenge:	Existing methods tend to select different demonstrations for each test instance, which is time-consuming and poses limitations in practical scenarios.
Approach:	They propose to select a representative subset of in-context demonstrations that can prompt different test instances in a specific task.
Outcome:	The proposed method can be used to generate representative in-context demonstrations.

Copied to clipboard

Necva Bölücü, Maciej Rybinski, Stephen Wan

Learning Task Representations from In-Context Learning (2025.findings-acl)

Challenge:	Prompt-based use of Large Language Models is becoming popular . specialized domains such as entity extraction are expensive to annotate .
Approach:	They propose to use a prompt set-up to provide training examples along with the inference request.
Outcome:	The proposed methods improve on a fully supervised transformer-based baseline.

Copied to clipboard

Baturay Saglam, Xinyang Hu, Zhuoran Yang, Dionysis Kalogerias, Amin Karbasi

Prompting Language Models for Linguistic Structure (2023.acl-long)

Challenge:	Existing methods for generalizing tasks to modalities beyond text fail to generalize effectively to linguistic tasks.
Approach:	They propose a method for encoding task information in ICL prompts as a function of attention heads within the transformer architecture.
Outcome:	The proposed method extracts task-specific information from in-context demonstrations and excels in both text and regression tasks.

Copied to clipboard

Terra Blevins, Hila Gonen, Luke Zettlemoyer

Towards Interpretable Mental Health Analysis with Large Language Models (2023.emnlp-main)

Challenge:	Existing prompting methods can test this hypothesis on autoregressive PLMs.
Approach:	They propose a structured prompting approach for linguistic structured prediction tasks that performs zero- and few-shot sequence tagging with autoregressive PLMs.
Outcome:	The proposed approach shows that the model can perform few-shot sequence tagging on part-of-speech taging, named entity recognition, and sentence chunking tasks.

Copied to clipboard

Kailai Yang, Shaoxiong Ji, Tianlin Zhang, Qianqian Xie, Ziyan Kuang, Sophia Ananiadou

LOFT: Scalable and More Realistic Long-Context Evaluation (2025.findings-naacl)

Challenge:	Existing studies on large language models lack adequate evaluations and prompting strategies for explainability.
Approach:	They evaluate the mental health analysis and emotional reasoning ability of large language models (LLMs) using 11 datasets across 5 tasks.
Outcome:	The proposed model shows strong in-context learning ability but still has a significant gap with advanced task-specific methods.

Copied to clipboard

Jinhyuk Lee, Anthony Chen, Zhuyun Dai, Dheeru Dua, Devendra Singh Sachan, Michael Boratko, Yi Luan, Séb Arnold, Vincent Perot, Siddharth Dalmia, Hexiang Hu, Xudong Lin, Panupong Pasupat, Aida Amini, Jeremy R. Cole, Sebastian Riedel, Iftekhar Naim, Ming-Wei Chang, Kelvin Guu

Mind Your Format: Towards Consistent Evaluation of In-Context Learning Improvements (2024.findings-acl)

Challenge:	Long-context language models (LCLMs) can be used to perform tasks traditionally reliant on external tools like retrieval systems or databases.
Approach:	They propose a benchmark to evaluate LCLMs' performance on in-context retrieval and reasoning tasks using a set of tokens.
Outcome:	The proposed model outperforms state-of-the-art retrieval and RAG systems on in-context retrieval tasks while still requiring prompting strategies.

Copied to clipboard

Anton Voronov, Lena Wolf, Max Ryabinin

Towards Diverse and Effective Question-Answer Pair Generation from Children Storybooks (2023.findings-acl)

Challenge:	Large language models demonstrate remarkable ability for learning to solve new tasks from a few examples.
Approach:	They propose to use templates to aggregate model predictions across multiple templates to improve model performance.
Outcome:	The proposed model ensembles boost model predictions while being robust to the choice of random set of templates.

Copied to clipboard

Sugyeong Eo, Hyeonseok Moon, Jinsung Kim, Yuna Hur, Jeongwook Kim, SongEun Lee, Changwoo Chun, Sungsoo Park, Heuiseok Lim

Natural Language Interfaces

Challenge:	Recent advances in QA pair generation (QAG) have raised interest in applying this technique to the educational field.
Approach:	They propose a QAG framework that enhances QA type diversity by producing different interrogative sentences and implicit/explicit answers.
Outcome:	The proposed framework outperforms state-of-the-art methods by significant margins, achieving improved diversity and quality.

On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model (2022.naacl-main)

Copied to clipboard

Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woomyoung Park, Jung-Woo Ha, Nako Sung

Dialogue Summarization with Mixture of Experts based on Large Language Models (2024.acl-long)

Challenge:	Recent studies on large-scale in-context language models have reported successful in-const zero- and few-shot learning ability.
Approach:	They investigate the effects of the pretraining corpus on in-context learning in a Korean-centric model.
Outcome:	The study shows that pretraining corpus size does not determine in-context learning ability . the findings suggest that in-constext learning is not always competitive .

Copied to clipboard

Yuanhe Tian, Fei Xia, Yan Song

Dialogue Systems & Conversational Agents

Challenge:	Existing studies for dialogue summarization use one model at a time or treat it as a black box.
Approach:	They propose an LLM-based approach with role-oriented routing and fusion generation to utilize mixture of experts for dialogue summarization.
Outcome:	The proposed approach produces informative and accurate dialogue summarization on widely used datasets.

Few-shot In-context Learning on Knowledge Base Question Answering (2023.acl-long)

Copied to clipboard

Tianle Li, Xueguang Ma, Alex Zhuang, Yu Gu, Yu Su, Wenhu Chen

In-Context Learning (and Unlearning) of Length Biases (2025.naacl-long)

Challenge:	KB-BINDER enables few-shot in-context learning over knowledge base questions . KBQA is a difficult problem due to the heterogeneity of knowledge bases .
Approach:	They propose a framework that enables few-shot in-context learning over KBQA tasks.
Outcome:	The proposed framework can outperform state-of-the-art models on GraphQA and MetaQA datasets.

Copied to clipboard

Stephanie Schoch, Yangfeng Ji

Revealing the Attention Floating Mechanism in Masked Diffusion Models (2026.findings-acl)

Challenge:	Existing work has demonstrated the ability of large language models to learn lexical and label biases in-context negatively impacts performance and robustness of models.
Approach:	They investigate the impact of length biases on in-context learning by analyzing model length information in-constext.
Outcome:	The proposed model learns length biases in the context window without parameter updates.

Copied to clipboard

Xin Dai, Pengcheng Huang, Zhenghao Liu, Shuo Wang, Yukun Yan, Chaojun Xiao, Yu Gu, Ge Yu, Maosong Sun

Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models (2025.naacl-long)

Challenge:	Masked diffusion models (MDMs) leverage bidirectional attention and a denoising process.
Approach:	They investigate the attention behaviors of Masked diffusion models by revealing the phenomenon of Attention Floating.
Outcome:	The proposed model doubles the performance of autoregressive models in knowledge-intensive tasks.

Copied to clipboard

Aliakbar Nafar, K. Brent Venable, Parisa Kordjamshidi

Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu (2025.acl-long)

Challenge:	Existing studies on in-context learning mechanisms are not consistent . current research identifies two main approaches to explain the ICL mechanism .
Approach:	They propose a framework for evaluating in-context learning mechanisms by focusing on regression tasks.
Outcome:	The proposed framework can solve regression problems and then measure the extent to which the LLM retrieves its internal knowledge versus learning from in-context examples.

Copied to clipboard

Renhao Pei, Yihong Liu, Peiqin Lin, François Yvon, Hinrich Schuetze

An Empirical Study of In-context Learning in LLMs for Machine Translation (2024.findings-acl)

Challenge:	In-context machine translation (MT) with large language models can take advantage of linguistic resources such as grammar books and dictionaries.
Approach:	They propose to use in-context machine translation (MT) with large language models to take advantage of linguistic resources such as grammar books and dictionaries.
Outcome:	The proposed approach can take advantage of dictionaries and grammar books, but its performance is poor for many lowresource languages.

Copied to clipboard

Pranjal Chitale, Jay Gala, Raj Dabre

Do Emergent Abilities Exist in Quantized Large Language Models: An Empirical Study (2024.lrec-main)

Challenge:	Recent studies focus on optimizing translation quality, with limited attention to understanding specific aspects of ICL that influence the said quality.
Approach:	They conduct the first of its kind, exhaustive study of in-context learning for machine translation (MT) they establish that ICL is primarily example-driven and not instruction-driven .
Outcome:	The proposed model is based on examples and not instruction-driven learning.

Copied to clipboard

Peiyu Liu, Zikang Liu, Ze-Feng Gao, Dawei Gao, Wayne Xin Zhao, Yaliang Li, Bolin Ding, Ji-Rong Wen

Can Large Language Models Follow Concept Annotation Guidelines? A Case Study on Scientific and Financial Domains (2024.findings-acl)

Challenge:	Large Language Models (LLMs) require significant computational resources for deployment and use.
Approach:	They propose to use low-bit quantization methods to reduce memory footprint and increase inference rate to improve performance of Large Language Models.
Outcome:	The proposed methods can reduce the memory footprint and increase the inference rate of LLMs.

Copied to clipboard

Marcio Fonseca, Shay Cohen

ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training (2024.acl-long)

Challenge:	Large language models (LLMs) are known to distill knowledge from vast datasets during the pretraining phase.
Approach:	They propose to use in-context concept definitions as prompts for zero-shot sentence labeling tasks.
Outcome:	The proposed model outperforms the leading open-source models and their proprietary APIs in most cases.

Copied to clipboard

Le Zhuo, Zewen Chi, Minghao Xu, Heyan Huang, Jianan Zhao, Heqi Zheng, Conghui He, Xian-Ling Mao, Wentao Zhang

PIAST: Rapid Prompting with In-context Augmentation for Scarce Training data (2026.acl-long)

Challenge:	Experimental results demonstrate that ProtLLM achieves superior performance against protein-specialized baselines on protein-centric tasks and induces zero-shot and in-context learning capabilities on protein language tasks.
Approach:	They propose a cross-modal large language model (LLM) that can handle protein-centric and protein-language tasks by using a dynamic protein mounting mechanism.
Outcome:	The proposed model can predict proteins from a vast pool of candidates and can also predict natural language and biological papers.

Copied to clipboard

Paweł Batorski, Paul Swoboda

PICLe: Pseudo-annotations for In-Context Learning in Low-Resource Named Entity Detection (2025.naacl-long)

Challenge:	Existing prompt generation methods are impractical in time and data constrained settings.
Approach:	They propose a fast automatic prompt construction algorithm that augments human instructions by generating a small set of few shot examples.
Outcome:	The proposed method outperforms existing prompting methods on classification, simplification, and MedQA.

Copied to clipboard

Sepideh Mamooler, Syrielle Montariol, Alexander Mathis, Antoine Bosselut

Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deterioration (2024.findings-acl)

Challenge:	In-context learning is sensitive to the choice of demonstrations and can be used for tasks with few examples.
Approach:	They propose a framework for in-context learning with noisy, pseudo-annotated demonstrations . they annotate large quantities of demonstrations in a zero-shot first pass .
Outcome:	The proposed framework outperforms ICL on biomedical NED datasets with zero human-annotation.

Copied to clipboard

Kejuan Yang, Xiao Liu, Kaiwen Men, Aohan Zeng, Yuxiao Dong, Jie Tang

Multi-Dimensional Evaluation of Text Summarization with In-Context Learning (2023.findings-acl)

Challenge:	Existing methods for extending the maximum context lengths of language models are lacking a strong baseline for in-context few-shot classification and on more challenging Chain-of-Thought reasoning, such as HotpotQA, deteriorate question miscomprehension and false inference.
Approach:	They propose to harness window-wise attention and positional embedding techniques to extend the maximum context lengths of language models.
Outcome:	The proposed method is able to extend the maximum context lengths of language models, e.g., 2048 for LLaMA, by harnessing window-wise attention and positional embedding techniques.

Copied to clipboard

Sameer Jain, Vaishakh Keshava, Swarnashree Mysore Sathyendra, Patrick Fernandes, Pengfei Liu, Graham Neubig, Chunting Zhou

Knowledge Editing through Chain-of-Thought (2025.emnlp-main)

Challenge:	In-context learning-based evaluators are competitive with learned evaluation frameworks for text summarization tasks.
Approach:	They propose to use large language models as multi-dimensional evaluators using in-context learning to evaluate text summarization tasks.
Outcome:	The proposed frameworks are competitive with existing frameworks on relevance and factual consistency, the authors show .

Copied to clipboard

Changyue Wang, Weihang Su, Qingyao Ai, Yichen Tang, Yiqun Liu

In-context Examples Selection for Machine Translation (2023.findings-acl)

Challenge:	Existing knowledge editing methods focus on multi-hop QA tasks and require frequent retraining.
Approach:	They propose a new knowledge editing framework that updates large language models with new information to maintain their world knowledge without retraining.
Outcome:	The proposed method achieves state-of-the-art performance while offering superior generalization, effectiveness, and stability compared to existing methods.

Copied to clipboard

Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, Marjan Ghazvininejad

On the In-context Generation of Language Models (2024.emnlp-main)

Challenge:	Large-scale generative models can perform a wide range of NLP tasks using in-context learning.
Approach:	They aim to understand the properties of good in-context examples for machine translation in both in-domain and out-of-domain settings.
Outcome:	The proposed model outperforms a strong kNN-MT baseline in 2 out of 4 out-of-domain datasets.

Copied to clipboard

Zhongtao Jiang, Yuanzhe Zhang, Kun Luo, Xiaowei Yuan, Jun Zhao, Kang Liu

AM2iCo: Evaluating Word Meaning in Context across Low-Resource Languages with Adversarial Examples (2021.emnlp-main)

Challenge:	Large language models (LLMs) have the ability of in-context generation (ICG) when given an in-text prompt, they can implicitly recognize the pattern of the examples and complete the prompt in the desired way.
Approach:	They propose a plausible latent variable model to model the distribution of pretrained corpora and formalize ICG as a problem of next topic prediction.
Outcome:	The proposed model can model the distribution of pretrained corpora and then formalize ICG as a problem of next topic prediction.

Copied to clipboard

Qianchu Liu, Edoardo Maria Ponti, Diana McCarthy, Ivan Vulić, Anna Korhonen

Improving Instruct Models for Free: A Study on Partial Adaptation (2025.emnlp-main)

Challenge:	Existing multilingual evaluation datasets that evaluate lexical semantics "in-context" have various limitations, including limited coverage of high-resource languages and superficial cues.
Approach:	They propose to use a set of pretrained language models to evaluate lexical semantics in context.
Outcome:	The proposed set shows that current models lag behind human performance in interpreting word meaning in cross-lingual contexts.

Copied to clipboard

Ozan Irsoy, Pengxiang Cheng, Jennifer L Chen, Daniel Preotiuc-Pietro, Shiyue Zhang, Duccio Pappadopulo

Automatic Mathematic In-Context Example Generation for LLM Using Multi-Modal Consistency (2025.coling-main)

Challenge:	Instruct models are deemed superior and more usable but can be eroded by instruction tuning . a recent study shows that instruct models are better at following instructions than base models .
Approach:	They scale down the strength of instruction tuning to improve model performance . they show that reducing instruction tuning results in material improvement .
Outcome:	The proposed model improves on a few-shot in-context learning benchmark . but it loses some degree of its in-training ability .

Copied to clipboard

Jaeseong Lee, Wei Yang, Gopal Gupta, Shiyi Wei

Reasoning

Challenge:	Existing methods for in-context learning require annotated datasets, resulting in higher computational costs and lower quality examples.
Approach:	They propose a framework that automatically generates high-quality in-context examples to enhance LLMs’ mathematical reasoning.
Outcome:	Evaluated on four math problem datasets, the proposed framework outperforms baseline methods with LLM accuracy ranging from 87.0% to 99.3%.

Characterizing Mechanisms for Factual Recall in Language Models (2023.emnlp-main)

Copied to clipboard

Qinan Yu, Jack Merullo, Ellie Pavlick

How to Make LLMs Forget: On Reversing In-Context Knowledge Edits (2025.naacl-long)

Challenge:	Language Models often integrate facts they memorized with new information that appears in a given context, causing competition within the model.
Approach:	They investigate distributional and mechanistic determinants of LM behavior in a dataset that queries for knowledge of world capitals . they use head attribution to identify individual attention heads that either promote the memorized answer or the in-context answer in the logits .
Outcome:	The proposed method can increase the rate of generating the in-context answer to 88% of the time by scaling up or down the value vector of individual attention heads at runtime.

Copied to clipboard

Paul Youssef, Zhixue Zhao, Jörg Schlötterer, Christin Seifert

What Do Language Models Learn in Context? The Structured Task Hypothesis. (2024.acl-long)

Challenge:	In-context knowledge editing (IKE) is an efficient and efficient knowledge editing method (Zheng et al., 2022b; Gangadhar and Stratos, 2024) it can be misused to manipulate responses opaquely, e.g., insert misinformation or offensive content.
Approach:	They propose to detect and reverse IKE-edits using only the top-10 output probabilities of the next token, even in a black-box setting.
Outcome:	The proposed method can be detected with high accuracy even in a black-box setting, achieving over 80% accuracy in recovering original, unedited outputs across multiple LLMs.

Copied to clipboard

Jiaoda Li, Yifan Hou, Mrinmaya Sachan, Ryan Cotterell

Language Models Struggle to Use Representations Learned In-Context (2026.acl-long)

Challenge:	Pre-trained large language models have exhibited an impressive ability to learn in context across various domains, e.g., code generation, education, medicine and even medicine.
Approach:	They taxonomize existing candidate theories into three competing hypotheses that explain LLMs’ ability to learn in context.
Outcome:	The proposed model can learn a task from in-context examples presented in a demonstration and generalize it to the prompt.

Copied to clipboard

Michael A. Lepori, Tal Linzen, Ann Yuan, Katja Filippova

Experimental Contexts Can Facilitate Robust Semantic Property Inference in Language Models, but Inconsistently (2024.emnlp-main)

Challenge:	a recent study shows that large language models are capable of inducing rich representations of data that are seen in-context . a novel task, adaptive world modeling, shows that even the most performant LLMs cannot reliably leverage novel semantics defined in-constitut.
Approach:	They propose to use in-context representations to induce rich representations of data . they also propose to probe models using a novel task to enable flexible deployment .
Outcome:	The proposed model can use in-context representations to complete simple downstream tasks.

Copied to clipboard

Kanishka Misra, Allyson Ettinger, Kyle Mahowald

MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers (2026.acl-long)

Challenge:	Recent zero-shot evaluations have highlighted important limitations in the abilities of language models (LMs) to perform meaning extraction.
Approach:	They propose to use in-context examples and instructions to improve LMs' robustness in performing property inheritance.
Outcome:	The proposed model can perform non-trivial property inheritance on in-context examples and instructions, but it is inconsistent with the task.

Copied to clipboard

Linrui Ma, Chun Hei Lo, Xinyu Wang, Peng Lu, Xihao Yuan, Hanting Chen, Kai Han, Xinghao Chen, Chengjun Zhan, Hanlin xu, Yichun Yin, Lifeng Shang, Feng Wen, Boxing Chen, Yufei Cui

Large-scale Lifelong Learning of In-context Instructions and How to Tackle It (2023.acl-long)

Challenge:	Existing approaches to improve efficiency often enforce rigid structural constraints such as local attention windows.
Approach:	They propose a framework that augments sparse-attention mechanisms with dynamically integrated in-context information through an efficient retrieval system.
Outcome:	Empirical results show that MATCH significantly improves the performance of sparse-attention models on synthetic and real-world natural-language tasks.

Copied to clipboard

Jisoo Mok, Jaeyoung Do, Sungjin Lee, Tara Taghavi, Seunghak Yu, Sungroh Yoon

CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations (2025.acl-long)

Challenge:	In-context instruction learning is a method to improve the target PLM’s instance- and task-level generalization performance as it observes more tasks.
Approach:	They propose to fine-tune a Pre-trained Language Model (PLM) on a set of tasks with in-context instructions and to extend this property to a scenario in which tasks are fed to the target PLM in a sequential manner.
Outcome:	The proposed method achieves noticeable improvements in both types of generalization, nearly reaching the upper bound performance obtained through joint training.

Copied to clipboard

Vignesh Kothapalli, Hamed Firooz, Maziar Sanjabi

Explanation-aware Soft Ensemble Empowers Large Language Model In-context Learning (2024.acl-long)

Challenge:	In-context learning and CoT are still poorly understood, but the precise mechanisms and architectural factors driving ICL and Co T are still unclear.
Approach:	They propose a framework and methodology to generate synthetic tokenized datasets and study chain-of-thought (CoT) in-context learning in language models.
Outcome:	The proposed framework and methodology allows fine grained control over the complexity of in-context examples by decoupling causal structure from underlying token processing functions.

Copied to clipboard

Yue Yu, Jiaming Shen, Tianqi Liu, Zhen Qin, Jing Nathan Yan, Jialu Liu, Chao Zhang, Michael Bendersky

Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoning (2024.findings-acl)

Challenge:	Recent advances in natural language processing (NLP) have witnessed the remarkable capabilities of Large Language Models (LLMs).
Approach:	They propose an Explanation-Aware Soft Ensemble framework to empower in-context learning with Large language models.
Outcome:	The proposed framework can be used to enhance in-context learning on seven natural language understanding tasks and four varying-size LLMs.

Copied to clipboard

Zhouhang Xie, Bodhisattwa Prasad Majumder, Mengjie Zhao, Yoshinori Maeda, Keiichi Yamada, Hiromi Wakaki, Julian McAuley

Reasoning

Challenge:	Motivational Interviewing (MI) requires a system that can infer how to motivate users to adopt positive lifestyle changes.
Approach:	They propose a framework that can learn and apply conversation strategies from expert demonstrations by using natural language inductive rules.
Outcome:	The proposed framework outperforms in-context demonstrations that are over 50 times longer and can learn natural language strategies from demonstrations.

Mitigating Label Biases for In-context Learning (2023.acl-long)

Copied to clipboard

Yu Fei, Yifan Hou, Zeming Chen, Antoine Bosselut

Shaking Up VLMs: Comparing Transformers and Structured State Space Models for Vision & Language Modeling (2024.emnlp-main)

Challenge:	Existing methods to categorize label biases in in-context learning (ICL) have not addressed all three types of label bias.
Approach:	They propose a method that estimates a language model’s label bias using random in-domain words from the task corpus to categorize and detect label biases in ICL.
Outcome:	The proposed method significantly improves the performance of GPT-J and GPT-3 on a wide range of tasks.

Copied to clipboard

Georgios Pantazopoulos, Malvina Nikandrou, Alessandro Suglia, Oliver Lemon, Arash Eshghi

ConstraintLLM: A Neuro-Symbolic Framework for Industrial-Level Constraint Programming (2025.emnlp-main)

Challenge:	a task-agnostic visual encoding yields minimal performance gains on grounding, but Transformers outperform Mamba at in-context multimodal retrieval.
Approach:	They propose to replace Transformers in Visual Language Models with Mamba, a structured state space model that demonstrates promising performance in sequence modeling.
Outcome:	The proposed model outperforms Transformers-based models in captioning, question answering, and reading comprehension.

Copied to clipboard

Weichun Shi, Minghao Liu, Wanting Zhang, Langchen Shi, Fuqi Jia, Feifei Ma, Jian Zhang

LLM as a metric critic for low resource relation identification (2024.findings-emnlp)

Challenge:	Constraint programming (CP) is a powerful paradigm for solving constraint optimization problems.
Approach:	They propose to use an open-source LLM to generate formal modeling for COPs.
Outcome:	The proposed model outperforms the baselines on the new IndusCP benchmark by 2x.

Copied to clipboard

Zhe Yang, Yi Huang, Yaqin Chen, Xiaoting Wu, Junlan Feng, Chao Deng

Learning Fine-Grained Grounded Citations for Attributed Large Language Models (2024.findings-acl)

Challenge:	Existing studies show that small language models (SLMs) overfit in low resource situations . however, the gap between pre-training and fine-tuning leads to performance decay .
Approach:	They propose to combine large language models and LLM for relation identification by co-evolution . they propose to use a masked language model prompt to generate a relation identification task .
Outcome:	The proposed model can handle low resource relation identification tasks with minimal overfitting . the proposed model provides essential background knowledge to assist training process .

Copied to clipboard

Lei Huang, Xiaocheng Feng, Weitao Ma, Yuxuan Gu, Weihong Zhong, Xiachong Feng, Weijiang Yu, Weihua Peng, Duyu Tang, Dandan Tu, Bing Qin

Decoding by Contrasting Knowledge: Enhancing Large Language Model Confidence on Edited Facts (2025.acl-long)

Challenge:	despite impressive performance, large language models still struggle with hallucinations . current approaches suffer from suboptimal citation quality due to reliance on in-context learning .
Approach:	They propose a framework that teaches large language models to generate fine-grained citations.
Outcome:	The proposed framework outperforms all baselines on the ALCE benchmark and achieves an average improvement of 14.21% in citation quality.

Copied to clipboard

Baolong Bi, Shenghua Liu, Lingrui Mei, Yiwei Wang, Junfeng Fang, Pengliang Ji, Xueqi Cheng

Guideline Learning for In-Context Information Extraction (2023.emnlp-main)

Challenge:	In-context knowledge editing (ICE) is currently the most effective method for knowledge editing, but it is constrained by the black-box modeling of LLMs and lacks interpretability.
Approach:	They propose a method to decode new knowledge by comparing logits with unedited knowledge to improve the accuracy of LLMs.
Outcome:	The proposed method improves the performance of LLaMA3-8B-instruct on MQuAKE by up to 219%.

Copied to clipboard

Chaoxu Pang, Yixuan Cao, Qiang Ding, Ping Luo

Information Extraction & Text Mining

Challenge:	Large language models can perform a task by conditioning on task instructions and a few input-output examples without optimizing any parameters.
Approach:	They propose a guideline learning framework for In-context IE which reflectively learns and follows guidelines.
Outcome:	The proposed framework improves the performance of in-context IE by synthesizing and following guidelines.

Inference and Verbalization Functions During In-Context Learning (2024.findings-emnlp)

Copied to clipboard

Junyi Tao, Xiaoyin Chen, Nelson Liu

Learning API Functionality from In-Context Demonstrations for Tool-based Agents (2025.findings-emnlp)

Challenge:	Previous work has found that, in some settings, ICL performance is minimally affected by using demonstrations with irrelevant label words.
Approach:	They hypothesize that large language models (LMs) perform in-context learning from a handful of demonstrations via two sequential processes: an inference function that solves the task and a verbalization function that maps the inferred answer to the label space.
Outcome:	The proposed model can be localized in specific layers across open-source models, including GEMMA-7B, MISTRAL-7B-V0.3, GEIMA-2-27B, and LLAMA-3.1-70B.

Copied to clipboard

Bhrij Patel, Ashish Jagmohan, Aditya Vempaty

Challenge:	Documentation is often missing, outdated, privatized, or inconsistent in tool-based agents.
Approach:	They propose a task that allows agents to learn API functionality directly from in-context demonstrations.
Outcome:	The proposed model improves the task success rate of an API-based agent by providing explicit function calls and natural language critiques.

In-Context Compositional Generalization for Large Vision-Language Models (2024.emnlp-main)

Copied to clipboard

Chuanhao Li, Chenchen Jing, Zhen Li, Mingliang Zhai, Yuwei Wu, Yunde Jia

COM-BOM: Bayesian Exemplar Search for Efficiently Exploring the Accuracy-Calibration Pareto Frontier (2025.emnlp-main)

Challenge:	Recent work shows that in-context learning for large language models exhibits compositional generalization capacity.
Approach:	They propose a method to exhibit in-context compositional generalization in large vision-language models by combining visual and linguistic modalities.
Outcome:	The proposed method reduces redundancy and complexity in in-context learning with LVLMs.

Copied to clipboard

Gaoxiang Luo, Aryan Deshwal

PerDucer: Keyphrase-Driven Personalization Inducer for Summarization from User Histories (2026.findings-acl)

Challenge:	Prior exemplar selection methods focus on maximizing predictive accuracy, neglecting model calibration.
Approach:	They propose to use a Bayesian optimization algorithm to optimize for predictive accuracy and calibration.
Outcome:	The proposed algorithm beats or matches baselines on multiple tasks from un-saturated MMLU-pro benchmarks while requiring minimal number of API calls.

Copied to clipboard

Parthiv Chatterjee, Asish Joel Batha, Sourish Dasgupta, Tanmoy Chakraborty

Summarization

Challenge:	Prior work reported that prepending long interaction histories to LLMs leads to unstable personalization, especially for multi-aspect documents.
Approach:	They propose a personalization inducer for frozen language models that maps latent preference signals to a small set of personalized keyphrases for the query document.
Outcome:	The proposed model outperforms the strongest history-prompting LLMs and SLMs in the PENS and OpenAI-Reddit benchmarks.

Measuring Pointwise 𝒱-Usable Information In-Context-ly (2023.findings-emnlp)

Copied to clipboard

Sheng Lu, Shan Chen, Yingya Li, Danielle Bitterman, Guergana Savova, Iryna Gurevych

Challenge:	In-context learning (ICL) is a new learning paradigm that has gained popularity along with the development of large language models.
Approach:	They propose to adapt a recently proposed hardness metric, pointwise V-usable information (PVI), to an in-context version.
Outcome:	The proposed hardness metric is compared with the original model and is more efficient because it requires only a few exemplars and does not require fine-tuning.

StateX: Enhancing RNN Recall via Post-training State Expansion (2026.findings-acl)

Copied to clipboard

Xingyu Shen, Yingfa Chen, Zhen Leng Thai, Xu Han, Zhiyuan Liu, Maosong Sun

Improving Context Fidelity via Native Retrieval-Augmented Reasoning (2025.emnlp-main)

Challenge:	Existing studies show that RNNs with large recurrent states are expensive to train . however, the ability to recall contextual information from long contexts is underperforms them in certain aspects.
Approach:	They propose a framework that expands the states of pre-trained RNNs by scaling them up to 1.3B . they use a recurrent architecture that compresses contextual information into a fixedsize state .
Outcome:	Experiments on models with up to 1.3B parameters show that StateX expands state sizes without incurring high post-training costs or compromising other capabilities.

Copied to clipboard

Suyuchen Wang, Jinlin Wang, Xinyu Wang, Shiqi Li, Xiangru Tang, Sirui Hong, Xiao-Wen Chang, Chenglin Wu, Bang Liu

Reasoning

Challenge:	Existing approaches to fidelity to contexts rely on expensive supervised fine-tuning to generate evidence post-answer or train models to perform web searches without improving utilization of the given context.
Approach:	They propose a native retrieval-augmented reasoning framework that integrates in-context evidence with the model’s own retrieval capabilities.
Outcome:	The proposed approach outperforms supervised fine-tuning, retrieval-augmented generation methods, and external retrieval solutions on multiple real-world and counterfactual QA benchmarks.

Bayesian Example Selection Improves In-Context Learning for Speech, Text and Visual Modalities (2024.emnlp-main)

Copied to clipboard

Siyin Wang, Chao-Han Yang, Ji Wu, Chao Zhang

RADCoT: Retrieval-Augmented Distillation to Specialization Models for Generating Chain-of-Thoughts in Query Expansion (2024.lrec-main)

Challenge:	Large language models (LLMs) can adapt to new tasks easily and efficiently in a training-free manner.
Approach:	They propose to use eBayesian in-context example selection method to extend the inference probability conditioned on in-constitut examples based on Bayes’ theorem to select in-strategy examples . Experimental results show the efficacy and robustness of their method on various models, tasks and modalities.
Outcome:	The proposed method is based on the eBayesian in-context example selection approach.

Copied to clipboard

Sung-Min Lee, Eunhwan Park, DongHyeon Jeon, Inho Kang, Seung-Hoon Na

Compositional Translation: A Novel LLM-based Approach for Low-resource Machine Translation (2025.findings-emnlp)

Challenge:	Large language models (LLMs) have demonstrated superior performance to that of small language models in information retrieval for various subtasks including dense retrieval, reranking, query expansion, and pseudo-document generation.
Approach:	They propose a retrieval-augmented model specialization that distills the capability of LLMs to generate the chain-of-thoughts (CoT) for query expansion into a RADCoT.
Outcome:	The proposed model can generate the chain-of-thoughts (CoT) for query expansion, reducing the burden of internalizing and retaining world knowledge in model parameters.

Copied to clipboard

Armel Randy Zebaze, Benoît Sagot, Rachel Bawden

Text Generation

Challenge:	generative large language models (LLMs) can perform in-context learning . machine translation (MT) has been shown to benefit from in-constitu examples .
Approach:	They propose a compositional translation paradigm that replaces naive few-shot MT with similarity-based demonstrations.
Outcome:	The proposed paradigm replaces naive few-shot MT with similarity-based demonstrations.

S3Prompt: Instructing the Model with Self-calibration, Self-recall and Self-aggregation to Improve In-context Learning (2024.lrec-main)

Copied to clipboard

Junda Chen, Jianting Liu

Self-Improvement Programming for Temporal Knowledge Graph Question Answering (2024.lrec-main)

Challenge:	Large language models have limitations in practical applications, such as unsupervised generation and recall of in-context examples.
Approach:	They propose a self-calibration, self-recall and self-aggregation prompt pipeline to solve these problems.
Outcome:	The proposed pipeline improves the performance of large language models without annotating datasets and model parameter updates.

Copied to clipboard

Zhuo Chen, Zhao Zhang, Zixuan Li, Fei Wang, Yutao Zeng, Xiaolong Jin, Yongjun Xu

Beyond Generation: Leveraging LLM Creativity to Overcome Label Bias in Classification (2025.findings-acl)

Challenge:	Existing methods implicitly model time constraints by learning time-aware embeddings of questions and candidate answers, which is far from understanding the question comprehensively.
Approach:	They propose a temporal-based temporal programming method that leverages the in-context learning ability of Large Language Models to understand combinatory time constraints in questions.
Outcome:	The proposed method outperforms existing methods on multiTQ and CronQuestions datasets and is highly efficient on multi-level questions.

Copied to clipboard

Xiaoyue Wang, Xin Liu

ICL CIPHERS: Quantifying ”Learning” in In-Context Learning via Substitution Ciphers (2025.emnlp-main)

Challenge:	Existing methods to mitigate label bias by leveraging in-domain data are often unavailable in real-world scenarios.
Approach:	They propose a calibration method that generates synthetic in-domain data from a few in-context demonstrations and utilizes it for calibration.
Outcome:	The proposed method reduces label bias by leveraging in-domain data from demonstrations.

Copied to clipboard

Zhouxiang Fang, Aayush Mishra, Muhan Gao, Anqi Liu, Daniel Khashabi

LLM×MapReduce: Simplified Long-Sequence Processing using Large Language Models (2025.acl-long)

Challenge:	Recent studies suggest that In-Context Learning operates in dual modes . however, disentangling these modes remains a challenging goal .
Approach:	They propose a class of task reformulations based on substitution ciphers borrowed from classic cryptography.
Outcome:	The proposed model can solve tasks with a BIJECTIVE mapping, but it requires 'deciphering' the latent cipher.

Copied to clipboard

Zihan Zhou, Chong Li, Xinyi Chen, Shuo Wang, Yu Chao, Zhili Li, Haoyu Wang, Qi Shi, Zhixing Tan, Xu Han, Xiaodong Shi, Zhiyuan Liu, Maosong Sun

Exploring In-context Example Generation for Machine Translation (2025.findings-acl)

Challenge:	Existing studies have focused on extending the context length of large language models (LLMs) due to their quadratic computational complexity and a lack of high-quality long training examples, most LLMs are trained with a limited window size.
Approach:	They propose a training-free framework that enables large language models to effectively process long texts using a divide-and-conquer strategy for comprehensive document understanding.
Outcome:	The proposed framework outperforms open-source and commercial long-context LLMs and is compatible with several models.

Copied to clipboard

Dohyun Lee, Seungil Chad Lee, Chanwoo Yang, Yujin Baek, Jaegul Choo

Text Generation

Challenge:	Large language models (LLMs) have demonstrated strong performance across various tasks with just a few examples.
Approach:	They propose a method that generates in-context example pairs without external resources.
Outcome:	The proposed method builds upon two prior criteria, relevance and diversity, which have been highlighted as key factors for in-context example selection.

Evaluating Answer Leakage Robustness of LLM Tutors against Adversarial Student Attacks (2026.acl-long)

Copied to clipboard

Jin Zhao, Marta Knežević, Tanja Käser

Robustness in NLP

Challenge:	Large Language Models (LLMs) are increasingly used in education, yet their default usefulness conflicts with pedagogical principles.
Approach:	They propose an adversarial student agent that they fine-tune to jailbreak LLM-based tutors and propose a benchmark to evaluate tutor robustness.
Outcome:	The proposed model fine-tunes to jailbreak LLM-based tutors, and shows that they perform well under adversarial student attacks.

Scaling Performance and Low-Resource Annotation with Many-Shot In-Context Learning for Named Entity Recognition (2026.findings-acl)

Copied to clipboard

Qi Zhang, Fangping Lan, Cornelia Caragea, Longin Jan Latecki, Eduard Dragut

Towards Understanding the Relationship between In-context Learning and Compositional Generalization (2024.lrec-main)

Challenge:	Existing studies on ICL for Named Entity Recognition (NER) have mainly explored few-shot settings, but the potential of scaling to hundreds of demonstrations has not been thoroughly investigated.
Approach:	They evaluate various LLMs across multiple domains using hundreds of ICL examples and then assess the feasibility of using many-shot ICL as a data annotation framework.
Outcome:	The proposed framework can be scaled to hundreds of examples and annotate and refining data for low-resource NER tasks.

Copied to clipboard

Sungjun Han, Sebastian Padó

Embedding-based In-Context Prompt Training for Enhancing LLMs as Text Encoders (2026.findings-acl)

Challenge:	In-context learning is an inductive bias for compositional generalization, but many deep neural architectures struggle with this ability.
Approach:	They propose to force a causal Transformer to in-context learn to promote compositional generalization by using earlier examples to generalize to later ones.
Outcome:	The proposed model can solve 'ordinary' learning problems by utilizing earlier examples to generalize to later ones, i.e., in-context learning.

Copied to clipboard

Ailiang Lin, Zhuoyun Li, Keyu Mao, Kotaro Funakoshi, Manabu Okumura

No Need for Explanations: LLMs can implicitly learn from mistakes in-context (2025.emnlp-main)

Challenge:	Large language models (LLMs) have been widely explored for embedding generation.
Approach:	They propose an embedding-based in-context prompt training strategy that leverages in-constext learning to generate high-quality embeddables while reducing computational burden.
Outcome:	The proposed method surpasses models trained on publicly available retrieval data and achieves state-of-the-art embedding performance on the MTEB benchmark.

Copied to clipboard

Lisa Alazraki, Maximilian Mozes, Jon Ander Campos, Tan Yi-Chern, Marek Rei, Max Bartolo

Evaluating Large Language Models for Detecting Antisemitism (2025.emnlp-main)

Challenge:	Existing literature assumes that correct answers to large language models must be accompanied by comprehensive rationales to be helpful.
Approach:	They propose to show incorrect answers to Large Language Models (LLMs) as a popular strategy to improve their performance in reasoning-intensive tasks.
Outcome:	The proposed approach outperforms chain-of-thought prompting in math reasoning tasks.

Copied to clipboard

Jay Patel, Hrudayangam Mehta, Jeremy Blackburn

BioTool: A Comprehensive Tool-Calling Dataset for Enhancing Biomedical Capabilities of Large Language Models (2026.acl-long)

Challenge:	Detecting hateful content is a challenging and important problem, but it requires continuous training to adapt to the ever-changing landscape of social media.
Approach:	They evaluate eight open-source LLMs’ capability to detect antisemitic content . they use in-context definition as a policy guideline to explore different prompts .
Outcome:	The proposed model outperforms the fine-tuned model with Guided-CoT .

Copied to clipboard

Xin Gao, Ruiyi Zhang, Meixi Du, Peijia Qin, Pengtao Xie