In-Context Compositional Generalization for Large Vision-Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that in-context learning for large language models exhibits compositional generalization capacity. |
| Approach: | They propose a method to exhibit in-context compositional generalization in large vision-language models by combining visual and linguistic modalities. |
| Outcome: | The proposed method reduces redundancy and complexity in in-context learning with LVLMs. |
Similar Papers
How Do In-Context Examples Affect Compositional Generalization? (2023.acl-long)
Copied to clipboard
| Challenge: | In-context learning paradigms that focus on large corpus are limiting compositional generalization performance. |
| Approach: | They propose a test suite to investigate in-context compositional generalization . they propose to use examples that are structurally similar to the test case . |
| Outcome: | The proposed test suite investigates in-context compositional generalization performance . it finds that the performance can be affected by the selection of in-const examples . |
Visual In-Context Learning for Large Vision-Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to improve the performance of Large Visual Language Models (LVLMs) are limited by cross-modal interactions and representation disparities. |
| Approach: | They propose a Visual In-Context Learning method that retrieves images via a 'Retrieval & Rerank' paradigm and summarises images with task intent and task-specific visual parsing to compose language-based demonstrations that reduce token count. |
| Outcome: | The proposed method reduces token count and alleviates cross-modal interaction problem on visual reasoning datasets. |
Diverse Demonstrations Improve In-context Compositional Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | In-context learning has shown great success in i.i.d semantic parsing splits . however, in compositional generalization, selecting similar demonstrations is insufficient . |
| Approach: | They propose a method to select diverse demonstrations that collectively cover all the structures required in the output program and encourage the model to generalize to new structures from these demonstrations. |
| Outcome: | The proposed method improves performance across three compositional generalization datasets and finetuning. |
Skills-in-Context: Unlocking Compositionality in Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | eliciting compositional generalization capabilities in large language models is challenging for advanced LLMs because they lack foundational skills and compositional examples in the same prompt context. |
| Approach: | They propose to use compositional generalization capabilities in large language models to elicit compositional skills in a prompt context. |
| Outcome: | The proposed structure enables LLMs to tackle more challenging problems with as few as two exemplars and unlocks their latent potential. |
From Introspection to Best Practices: Principled Analysis of Demonstrations in Multimodal In-Context Learning (2025.naacl-long)
Copied to clipboard
| Challenge: | Motivated by in-context learning capabilities of Large Language Models (LLMs), multimodal LLMs with additional visual modality are also exhibited with similar ICL abilities when multiple image-text pairs are provided as demonstrations. |
| Approach: | They conduct systematic and principled evaluation of multimodal ICL for models of different scales on a broad spectrum of new yet critical tasks. |
| Outcome: | The proposed model performance improves on a broad spectrum of new yet critical tasks. |
Learning to Select In-Context Demonstration Preferred by Large Language Model (2025.findings-acl)
Copied to clipboard
| Challenge: | In-context learning (ICL) enables large language models to perform tasks with only a few examples as demonstrations. |
| Approach: | They propose a generative preference learning framework that leverages LLM feedback to directly optimize demonstration selection for ICL. |
| Outcome: | Experiments on 19 datasets across 11 task categories show that GenICL achieves superior performance than existing methods in selecting the most effective demonstrations. |
In-context Learning Generalizes, But Not Always Robustly: The Case of Syntax (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is a common method for teaching large language models new tasks . given labeled examples in the input context, the model learns to perform the task without weight updates. |
| Approach: | They examine whether models guided via ICL infer the underlying structure of the task defined by the context or rely on superficial heuristics that only generalize to identically distributed examples. |
| Outcome: | The proposed model generalizes syntactically or linearly on out-of-distribution examples . the proposed model is able to generalize better on pre-trained models . |
On the Additive Compositionality of Task Vectors in Vision–Language Models (2026.eacl-short)
Copied to clipboard
| Challenge: | In-context learning (ICL) in large language models (LLMs) has been shown to operate through task vectors, but its extension to vision-language models (VLMs) remains underexplored. |
| Approach: | They construct visual reasoning tasks with clearly defined subtasks and extract task vectors from few-shot demonstrations. |
| Outcome: | The proposed model can be extended to vision-language models (VLMs) by adding the vectors of its constituent subtasks. |
Generating Demonstrations for In-Context Compositional Generalization in Grounded Language Learning (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-Context-learning and few-shot prompting are viable methods for compositional output generation but they are sensitive to the choice of support examples. |
| Approach: | They propose a method which generates supports and targets current state of the world and then uses them in-context-learning to solve a query. |
| Outcome: | The proposed agent improves performance on a previously unsolved compositional generalization test without loss of performance in other areas. |
Exploring Compositional Generalization of Large Language Models (2024.naacl-srw)
Copied to clipboard
| Challenge: | a recent study has found that large language models can generalize compositional instructions from simple instructions to complex ones. |
| Approach: | They study the generalization ability of large language models with respect to compositional instructions . they first construct a dataset with the help of ChatGPT guided by the self-instruct technique . |
| Outcome: | The proposed model can generalize from simple instructions to more intricate ones, the authors show . their results show that training LLMs on higher-order compositional instructions improves performance on lower-order ones, but not on higher order ones. |