Chain-of-Quizzes: Pedagogy-inspired Example Selection in In-Context-Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a powerful tool for enhancing large language models (LLMs) by mimicking the human learning process. |
| Approach: | They propose a Chain-of-Quizzes framework that uses LLMs to answer a quiz to sift 'good' examples, combine them iteratively with the increasing complexity, and utilize a final exam to gauge the combined example chains. |
| Outcome: | The proposed framework outperforms baseline models on diverse reasoning datasets and shows that it is scalable and can be used in future research. |
Similar Papers
CoE-SQL: In-Context Learning for Multi-Turn Text-to-SQL with Chain-of-Editions (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have demonstrated that Large Language Models (LLMs) have impressive capabilities in a variety of domains and tasks. |
| Approach: | They propose a method which prompts LLMs to generate SQL queries based on the previously generated SQL query with an edition chain. |
| Outcome: | The proposed method outperforms different in-context learning baselines and achieves state-of-the-art performance on two benchmarks SParC and CoSQL using LLMs. |
Chain-of-Interactions: Multi-step Iterative ICL Framework for Abstractive Task-Oriented Dialogue Summarization of Conversational AI Interactions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have introduced paradigm-shifting approaches in natural language processing, yet their transformative in-context learning (ICL) capabilities remain underutilized, especially in customer service dialogue summarization. |
| Approach: | They propose a single-instance, multi-step framework that orchestrates information extraction, self-correction, and evaluation through sequential interactive generation chains. |
| Outcome: | The proposed framework outperforms existing models and prompts in the customer service dialogue summarization domain. |
In-Context Learning Creates Task Vectors (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a powerful new learning paradigm for Large Language Models (LLMs). |
| Approach: | They propose to use a model with a prompt and a query to learn a mapping based on two examples to produce the output. |
| Outcome: | The proposed model can learn functions from a simple structure based on a training set and a single task vector calculated from the training set. |
Se2: Sequential Example Selection for In-Context Learning (2024.findings-acl)
Copied to clipboard
| Challenge: | Prior work has explored the selection of examples for in-context learning, neglecting the internal relationships between examples and exist an inconsistency between training and inference. |
| Approach: | They propose a sequential-aware method that leverages the LLM’s feedback on varying context, aiding in capturing inter-relationships and sequential information among examples. |
| Outcome: | Experiments on 23 NLP tasks show that Se2 surpasses baselines and achieves 42% relative improvement over random selection. |
Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown impressive reasoning abilities when prompted with Chain-of-Thought (CoT). |
| Approach: | They propose to categorize Chain-of-X methods by taxonomies of nodes, i.e., the X in CoX, and application tasks, and then categorise them by taxanomies and discuss potential future directions. |
| Outcome: | The proposed methods are categorised by taxonomies of nodes, i.e., the X in CoX, and application tasks. |
Revisiting In-Context Learning with Long Context Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | In-Context Learning (ICL) is a technique by which language models make predictions based on examples provided in their input context. |
| Approach: | They revisited previous studies using in-context learning techniques . they found that using a data augmentation approach, they significantly improved ICL performance . |
| Outcome: | The proposed approach significantly improves ICL performance on 18 datasets spanning 4 tasks . the proposed approach does not improve performance over a simple random sample selection method . |
In-Context Learning with Iterative Demonstration Selection (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing literature has highlighted the importance of selecting examples that are diverse or semantically similar to the test sample . Existing studies have shown that the optimal selection dimension, i.e., diversity or similarity, is task-specific. |
| Approach: | They propose to use zero-shot chain-of-thought reasoning to iteratively select examples that are diverse but still strongly correlated with the test sample as ICL demonstrations. |
| Outcome: | The proposed method outperforms existing demonstration selection methods on reasoning, question answering, and topic classification tasks. |
Coverage-based Example Selection for In-Context Learning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a training-free paradigm of fewshot inference that can generalize to novel tasks by conditioning on a few task examples. |
| Approach: | They show that BERTScore-Recall (BSR) selects better examples that demonstrate more of the salient aspects of the test input. |
| Outcome: | The proposed model outperforms methods that leverage task or LLM-specific training on compositional tasks. |
CoT-ICL Lab: A Synthetic Framework for Studying Chain-of-Thought Learning from In-Context Demonstrations (2025.acl-long)
Copied to clipboard
| Challenge: | In-context learning and CoT are still poorly understood, but the precise mechanisms and architectural factors driving ICL and Co T are still unclear. |
| Approach: | They propose a framework and methodology to generate synthetic tokenized datasets and study chain-of-thought (CoT) in-context learning in language models. |
| Outcome: | The proposed framework and methodology allows fine grained control over the complexity of in-context examples by decoupling causal structure from underlying token processing functions. |
Learning vs Retrieval: The Role of In-Context Examples in Regression with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing studies on in-context learning mechanisms are not consistent . current research identifies two main approaches to explain the ICL mechanism . |
| Approach: | They propose a framework for evaluating in-context learning mechanisms by focusing on regression tasks. |
| Outcome: | The proposed framework can solve regression problems and then measure the extent to which the LLM retrieves its internal knowledge versus learning from in-context examples. |