| Challenge: | Existing approaches to improve the performance of Large Visual Language Models (LVLMs) are limited by cross-modal interactions and representation disparities. |
| Approach: | They propose a Visual In-Context Learning method that retrieves images via a 'Retrieval & Rerank' paradigm and summarises images with task intent and task-specific visual parsing to compose language-based demonstrations that reduce token count. |
| Outcome: | The proposed method reduces token count and alleviates cross-modal interaction problem on visual reasoning datasets. |
Similar Papers
In-Context Compositional Generalization for Large Vision-Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that in-context learning for large language models exhibits compositional generalization capacity. |
| Approach: | They propose a method to exhibit in-context compositional generalization in large vision-language models by combining visual and linguistic modalities. |
| Outcome: | The proposed method reduces redundancy and complexity in in-context learning with LVLMs. |
A Survey on In-context Learning (2024.emnlp-main)
Copied to clipboard
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Baobao Chang, Xu Sun, Lei Li, Zhifang Sui
| Challenge: | In-context learning (ICL) is a new paradigm for natural language processing . large language models (LLMs) demonstrate the ability to learn from a few examples . |
| Approach: | They propose to explore ICL to evaluate and extrapolate the ability of large language models. |
| Outcome: | The proposed methods can be used to evaluate and extrapolate the ability of large language models. |
Revisiting In-Context Learning with Long Context Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) is a technique by which language models make predictions based on examples provided in their input context. |
| Approach: | They revisited previous studies using in-context learning techniques . they found that using a data augmentation approach, they significantly improved ICL performance . |
| Outcome: | The proposed approach significantly improves ICL performance on 18 datasets spanning 4 tasks . the proposed approach does not improve performance over a simple random sample selection method . |
MetaVL: Transferring In-Context Learning Ability From Language Models to Vision-Language Models (2023.acl-short)
Copied to clipboard
| Challenge: | Large-scale pre-trained vision-language models do not possess the ability to conduct in-context learning. |
| Approach: | They propose to meta-train a language model to perform in-context learning on NLP tasks and then transfer this model to VL tasks by attaching a visual encoder. |
| Outcome: | The proposed model outperforms the baseline model on VQA, OK-VQA, and GQA while having 20 times fewer parameters. |
On Many-Shot In-Context Learning for Long-Context Evaluation (2025.acl-long)
Copied to clipboard
| Challenge: | Existing benchmarks primarily evaluate long-context language models' retrieval capabilities. |
| Approach: | They propose a benchmark to evaluate long-context language models' retrieval capabilities by using MANYICLBENCH. |
| Outcome: | The proposed model performs better with additional demonstrations than translation and reasoning tasks. |
In-Context Learning with Long-Context Models: An In-Depth Exploration (2025.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is limited by context length, but it can be used for many tasks. |
| Approach: | They study the behavior of in-context learning at an extreme context length . example retrieval shows excellent performance at low context lengths but has diminished gains . |
| Outcome: | The proposed model can perform many tasks with reasonable accuracy when a few examples are provided in-context. |
DRUM: Learning Demonstration Retriever for Large MUlti-modal Models (2025.acl-srw)
Copied to clipboard
| Challenge: | Recent studies have shown that large language models (LLMs) have impressive capabilities in dealing with new tasks with the help of in-context learning (ICL). |
| Approach: | They propose to concate the image and text embeddings to enhance the retrieval performance of a visual-language task and to calculate a list-wise ranking loss for training the embeddable model. |
| Outcome: | The proposed framework fine-tunes the CLIP embedding model to better meet the needs of the large vision-language models. |
Focused Large Language Models are Stable Many-Shot Learners (2024.emnlp-main)
Copied to clipboard
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li
| Challenge: | In-Context Learning (ICL) enables large language models to achieve rapid task adaptation by learning from demonstrations. |
| Approach: | They propose a training-free method that disperses model attention from the query . they propose 'focus' search strategy that uses model perplexity to ensure sufficient attention . |
| Outcome: | The proposed method achieves an average performance improvement of 5.2% over vanilla ICL and scales well with many-shot demonstrations. |
Enhancing Advanced Visual Reasoning Ability of Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in Vision-Language (VL) research have sparked new benchmarks for complex visual reasoning, challenging models’ advanced reasoning ability. |
| Approach: | They propose a novel multi-modal in-context learning methodology to enhance LLMs’ contextual understanding and reasoning. |
| Outcome: | The proposed model achieves SOTA performance among all visual reasoning tasks and achieves a 'higher level of accuracy' than previous models. |
Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on in-context learning mechanisms are not consistent . current research identifies two main approaches to explain the ICL mechanism . |
| Approach: | They propose a framework for evaluating in-context learning mechanisms by focusing on regression tasks. |
| Outcome: | The proposed framework can solve regression problems and then measure the extent to which the LLM retrieves its internal knowledge versus learning from in-context examples. |