Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment (2023.acl-long)
Copied to clipboard
| Challenge: | a handful of studies have explored ICL in a cross-lingual setting . emergence of large-scale, pretrained, Transformer-based language models has marked the commencement of an avant-garde era in NLP. |
| Approach: | They propose a novel prompt construction strategy to bridge the gap between ICL and cross-lingual text classification. |
| Outcome: | The proposed approach outperforms random prompt selection by a large margin across three tasks using 44 different cross-lingual pairs. |
Similar Papers
LLMs Are Few-Shot In-Context Low-Resource Language Learners (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) empowers large language models to perform diverse tasks in underrepresented languages using only short in-contrast information. |
| Approach: | They extensively assess the effectiveness of in-context learning with LLMs in low-resource languages . they also identify the shortcomings of in context label alignment . |
| Outcome: | The proposed approach improves understanding quality of low-resource languages by closing the language gap in the target language. |
Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have transformed NLP with their remarkable In-context Learning capabilities. |
| Approach: | They propose to use large language models to generalize from labeled examples of predefined tasks to novel tasks . they use biological neurons and the Transformer architecture to study the potential for information sharing across tasks. |
| Outcome: | The proposed model can generalize from labeled examples of predefined tasks to novel tasks despite no examples from the target task in the context. |
From Cross-Task Examples to In-Task Prompts: A Graph-Based Pseudo-Labeling Framework for In-context Learning (2025.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) enables large language models to perform novel tasks without parameter updates by conditioning on a few input-output examples. |
| Approach: | They propose a cost-efficient two-stage pipeline that reduces reliance on LLMs for data labeling. |
| Outcome: | The proposed pipeline reduces reliance on LLMs for data labeling . it leverages readily available cross-task examples to prompt an LLM and pseudo-label a small set of target task instances. |
Blessing of Multilinguality: A Systematic Analysis of Multilingual In-Context Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a widely adopted technique for learning large language models . however, there is little systematic understanding of when and why it works well . |
| Approach: | They analyze multilingual in-context learning using demonstrations in HRLs to enhance cross-lingual transfer. |
| Outcome: | The proposed method outperforms English-only models on high-resource languages . the study shows that the presence of irrelevant non-English sentences in the prompt yields measurable gains . |
It’s All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Low-resource languages, especially those written in rare scripts, remain unsupported by large language models due to lack of training data. |
| Approach: | They evaluate 20 under-represented languages across three state-of-the-art multilingual LLMs and compare their methods to parameter-efficient fine-tuning. |
| Outcome: | The proposed methods compare with parameter-efficient fine-tuning (PEFT) on low-resource languages. |
In-context Mixing (ICM): Code-mixed Prompts for Multilingual LLMs (2024.acl-long)
Copied to clipboard
| Challenge: | In-context mixing is a prompting technique for effective in-contact learning with multilingual large language models. |
| Approach: | They propose a prompting technique called in-context mixing for effective in-constext learning with multilingual large language models. |
| Outcome: | The proposed prompts perform better with multilingual large language models. |
Enhancing Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies (2023.findings-emnlp)
Copied to clipboard
Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, Dragomir Radev
| Challenge: | In-context learning (ICL) is a new approach to natural language processing tasks that rely on large language models to make predictions based on context . recent studies have shown that neural symbolic design is the preferred choice for question answering systems because of its limited working memory and unreliable long-term memory. |
| Approach: | They propose to extend in-context learning to question answering tasks that utilize structured knowledge sources and to explore various prompt design strategies for employing LLMs. |
| Outcome: | The proposed approach outperforms the state-of-the-art system by 2.5 points and the best fine-tuned system by 5.1 points on the Spider dataset. |
More Samples or More Prompts? Exploring Effective Few-Shot In-Context Learning for LLMs with In-Context Sampling (2024.findings-naacl)
Copied to clipboard
Bingsheng Yao, Guiming Chen, Ruishi Zou, Yuxuan Lu, Jiachen Li, Shao Zhang, Yisi Sang, Sijia Liu, James Hendler, Dakuo Wang
| Challenge: | Existing studies on LLM prompting focus on selecting a better set of data samples inside one single prompt input, but why not design and leverage multiple ICL prompts together to further improve the LLM’s performance? |
| Approach: | They propose a low-resource LLM prompting technique to optimize the construction of multiple ICL prompt inputs to produce confident predictions. |
| Outcome: | The proposed technique can produce confident predictions by optimizing the construction of multiple ICL prompt inputs on four NLI datasets and one QA dataset. |
Why Multimodal In-Context Learning Lags Behind? Unveiling the Inner Mechanisms and Bottlenecks (2026.acl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) enables models to adapt to new tasks via inference-time demonstrations. |
| Approach: | They propose a simple inference-stage enhancement method that reinforces task mapping transfer. |
| Outcome: | The proposed method strengthens task mapping transfer in multimodal models . it performs comparable to text-only ICL in zero-shot settings but degrades significantly under few-shot demonstrations. |
Tutor-ICL: Guiding Large Language Models for Improved In-Context Learning Performance (2024.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning (ICL) is a dominant paradigm in natural language processing. |
| Approach: | They propose a prompting method for classification tasks using exemplar answers in a *comparative format' they also propose introducing a test instance before the exemplars to improve performance . |
| Outcome: | The proposed method achieves up to 13.76% increase in accuracy on classification tasks across decoder-only and encoder-decoder LLMs. |