Why Multimodal In-Context Learning Lags Behind? Unveiling the Inner Mechanisms and Bottlenecks (2026.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) enables models to adapt to new tasks via inference-time demonstrations. |
| Approach: | They propose a simple inference-stage enhancement method that reinforces task mapping transfer. |
| Outcome: | The proposed method strengthens task mapping transfer in multimodal models . it performs comparable to text-only ICL in zero-shot settings but degrades significantly under few-shot demonstrations. |
Similar Papers
From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning (2025.naacl-long)
Copied to clipboard
| Challenge: | Motivated by in-context learning capabilities of Large Language Models (LLMs), multimodal LLMs with additional visual modality are also exhibited with similar ICL abilities when multiple image-text pairs are provided as demonstrations. |
| Approach: | They conduct systematic and principled evaluation of multimodal ICL for models of different scales on a broad spectrum of new yet critical tasks. |
| Outcome: | The proposed model performance improves on a broad spectrum of new yet critical tasks. |
TACO: Enhancing Multimodal In-context Learning via Task Mapping-Guided Sequence Configuration (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multimodal in-context learning (ICL) is a key mechanism for harnessing the capabilities of large vision–language models. |
| Approach: | They propose a transformer-based model with task-aware attention that dynamically configures ICL sequences. |
| Outcome: | Experiments on five LVLMs and nine datasets show that TACO surpasses baselines across diverse ICL tasks. |
How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes (2024.naacl-short)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) for generalist models is a promising direction that offers transfer learning potential. |
| Approach: | They propose to combine multi-task learning (MTL) with in-context learning (ICL) to build models that can generalize to multiple tasks while being robust to out-of-distribution examples. |
| Outcome: | The proposed training strategies enable models to learn difficult tasks while mixing in prior tasks, denoted as mixed curriculum. |
In-Context Learning Creates Task Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a powerful new learning paradigm for Large Language Models (LLMs). |
| Approach: | They propose to use a model with a prompt and a query to learn a mapping based on two examples to produce the output. |
| Outcome: | The proposed model can learn functions from a simple structure based on a training set and a single task vector calculated from the training set. |
What In-Context Learning “Learns” In-Context: Disentangling Task Recognition and Task Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) can perform in-context learning (ICL) with only a few demonstrations, but its mechanisms are not well-understood. |
| Approach: | They characterize two ways in which LLMs leverage demonstrations to solve tasks with a few demonstrations. |
| Outcome: | The proposed model achieves non-trivial performance with only TR, and TR does not improve with larger models or more demonstrations. |
How Far Can In-Context Alignment Go? Exploring the State of In-Context Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that In-Context Learning (ICA) can align Large Language Models (LLMs) with human preferences without requiring parameter adjustments. |
| Approach: | They investigate the effectiveness of each part in enabling ICA to function effectively and examine how variants in these parts impact alignment performance. |
| Outcome: | The proposed model can comprehend human instructions without parameter adjustments. |
In-Context Learning Boosts Speech Recognition via Human-like Adaptation to Speakers and Language Varieties (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing models fail to adapt to unfamiliar speakers and language varieties . however, there are significant gaps in the adaptation of certain varieties based on the test speaker, variety, or recording conditions . |
| Approach: | They propose a framework that allows for in-context learning in Phi-4 Multimodal . they find that as few as 12 example utterances reduce word error rates by 19.7% . |
| Outcome: | The proposed framework reduces word error rates by 19.7% across diverse English corpora. |
Focused Large Language Models are Stable Many-Shot Learners (2024.emnlp-main)
Copied to clipboard
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li
| Challenge: | In-Context Learning (ICL) enables large language models to achieve rapid task adaptation by learning from demonstrations. |
| Approach: | They propose a training-free method that disperses model attention from the query . they propose 'focus' search strategy that uses model perplexity to ensure sufficient attention . |
| Outcome: | The proposed method achieves an average performance improvement of 5.2% over vanilla ICL and scales well with many-shot demonstrations. |
Label Words as Local Task Vectors in In-Context Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable abilities, one of the most important being in-context learning (ICL). |
| Approach: | They hypothesized that the network creates a task vector in specific positions during ICL, which can be computed by averaging across the dataset. |
| Outcome: | The proposed model can achieve zero-shot performance with dummy inputs comparable to few-shot learning by patching the global task vector. |
One Task Vector is not Enough: A Large-Scale Study for In-Context Learning (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing studies limit comprehensive analysis of large language models based on task vectors . recent work points to "task vectors" as mechanism for encoding task rules . |
| Approach: | They propose a novel task vector with 30 input-output pairs for in-context learning . they use a few prompt-based examples to adapt to new tasks without weight updates . |
| Outcome: | Experiments with Llama-3-8B on QAF show task vector performance peaks at intermediate layer . complex tasks rely on multiple, subtask-specific vectors rather than a single vector . |