From Fake to Real: Mitigating Out-of-Distribution Bias in In-Context Learning via Feedback Supervision from Large Language Models

Challenge:	In-Context Learning (ICL) is one of the most common methods for complex Natural Language Understanding tasks.
Approach:	They propose a method that uses model confidence and perturbation perplexity to enhance the quality of pseudo-labels.
Outcome:	The proposed method reduces OOD biases by avoiding direct use of source data.

Similar Papers

Overview

Challenge

Approach

Outcome

Beyond Generation: Leveraging LLM Creativity to Overcome Label Bias in Classification (2025.findings-acl)

Copied to clipboard

Xiaoyue Wang, Xin Liu

Information Retrieval

Challenge:	Existing methods to mitigate label bias by leveraging in-domain data are often unavailable in real-world scenarios.
Approach:	They propose a calibration method that generates synthetic in-domain data from a few in-context demonstrations and utilizes it for calibration.
Outcome:	The proposed method reduces label bias by leveraging in-domain data from demonstrations.

Mitigating Label Biases for In-context Learning (2023.acl-long)

Copied to clipboard

Yu Fei, Yifan Hou, Zeming Chen, Antoine Bosselut

Responsible & Trustworthy NLP

Challenge:	Existing methods to categorize label biases in in-context learning (ICL) have not addressed all three types of label bias.
Approach:	They propose a method that estimates a language model’s label bias using random in-domain words from the task corpus to categorize and detect label biases in ICL.
Outcome:	The proposed method significantly improves the performance of GPT-J and GPT-3 on a wide range of tasks.

Enhancing Input-Label Mapping in In-Context Learning with Contrastive Decoding (2025.acl-short)

Copied to clipboard

Keqin Peng, Liang Ding, Yuanxin Ouyang, Meng Fang, Yancheng Yuan, Dacheng Tao

Semantic Text Processing

Challenge:	Prior research has found that large language models overlook input-label mapping information in ICL, relying more on their pre-trained knowledge.
Approach:	They propose a novel method that contrasts input-label mappings between positive and negative in-context examples to improve model performance.
Outcome:	The proposed method improves performance on 7 natural language understanding tasks without additional training.

Generative Calibration for In-context Learning (2023.findings-emnlp)

Copied to clipboard

Zhongtao Jiang, Yuanzhe Zhang, Cao Liu, Jun Zhao, Kang Liu

Low-Resource NLP

Challenge:	In-context learning is one of the most exciting features of large language models . performance is sensitive to various configurations of the prompt, such as the choice or order of the training examples.
Approach:	They propose to calibrate the in-context predictive distribution by adjusting the label marginal . they find that the proposed method outperforms the ICL and state-of-the-art calibration methods .
Outcome:	The proposed method outperforms state-of-the-art methods by 27% absolute in macro-F1.

From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning (2025.findings-emnlp)

Copied to clipboard

Zihan Chen, Song Wang, Xingbo Fu, Chengshuai Shi, Zhenyu Lei, Cong Shen, Jundong Li

Low-Resource NLP

Challenge:	In-context learning (ICL) enables large language models to perform novel tasks without parameter updates by conditioning on a few input-output examples.
Approach:	They propose a cost-efficient two-stage pipeline that reduces reliance on LLMs for data labeling.
Outcome:	The proposed pipeline reduces reliance on LLMs for data labeling . it leverages readily available cross-task examples to prompt an LLM and pseudo-label a small set of target task instances.

Using Natural Language Explanations to Improve Robustness of In-context Learning (2024.acl-long)

Copied to clipboard

Xuanli He, Yuxiang Wu, Oana-Maria Camburu, Pasquale Minervini, Pontus Stenetorp

Responsible & Trustworthy NLP

Challenge:	Recent studies show that large language models excel in many tasks via in-context learning (ICL). However, ICL struggles to execute complex tasks such as arithmetic, commonsense, and symbolic reasoning.
Approach:	They propose to augment ICL with natural language explanations (NLEs) to produce further NLEs on adversarial datasets.
Outcome:	The proposed approach yields more accurate results than zero-shot-ICL and using only human-generated NLEs on eight adversarial datasets.

Beyond Performance: Quantifying and Mitigating Label Bias in LLMs (2024.naacl-long)

Copied to clipboard

Yuval Reif, Roy Schwartz

Responsible & Trustworthy NLP

Challenge:	Large language models exhibit undesirable preference toward predicting certain answers over others, despite their adaptability to diverse tasks.
Approach:	They propose a label bias calibration method that outperforms recent calibration approaches for improving performance and mitigating label bias.
Outcome:	The proposed method outperforms calibration approaches for improving performance and mitigating label bias.

LLMs are Better Than You Think: Label-Guided In-Context Learning for Named Entity Recognition (2025.emnlp-main)

Copied to clipboard

Fan Bai, Hamid Hassanzadeh, Ardavan Saeedi, Mark Dredze

Semantic Text Processing

Challenge:	Named Entity Recognition (NER) tasks are performed using only a few demonstrations.
Approach:	They propose a method that leverages training labels through token-level statistics to improve ICL performance.
Outcome:	The proposed method outperforms existing methods on five NER datasets and is robust in low-resource settings.

TABGEN-ICL: Residual-Aware In-Context Example Selection for Tabular Data Generation (2025.findings-acl)

Copied to clipboard

Liancheng Fang, Aiwei Liu, Hengrui Zhang, Henry Peng Zou, Weizhi Zhang, Philip S. Yu

Multimodality

Challenge:	Existing approaches to tabular data generation require fine-tuning, which is computationally expensive.
Approach:	They propose a new in-context learning framework to prompt a fixed LLM with in-constitut examples to enhance the in-text learning ability of LLMs for tabular data generation.
Outcome:	The proposed framework outperforms random selection strategies on five real-world tabular datasets and reduces error rate by 42.2% on fidelity metric.

Interpret and Improve In-Context Learning via the Lens of Input-Label Mappings (2025.acl-long)

Copied to clipboard

Chenghao Sun, Zhen Huang, Yonggang Zhang, Le Lu, Houqiang Li, Xinmei Tian, Xu Shen, Jieping Ye

Responsible & Trustworthy NLP

Challenge:	Large language models excel at downstream NLP tasks through in-context learning . however, the internal mechanisms behind ICL remain under-explored .
Approach:	They propose a PC patching approach to identify modules where input-label mappings function . they observe and verify that key heads utilize input-labeled mappings to generate target labels for new queries.
Outcome:	The proposed approach detects modules where input-label mappings function . it also detects that key heads use the mappings to generate labels for new queries .