Linguistic Neuron Overlap Patterns to Facilitate Cross-lingual Transfer on Low-resource Languages (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have focused on data-efficient methods, particularly Cross-lingual In-Context Learning (X-ICL) |
| Approach: | They propose a method to improve cross-lingual in-context learning for low-resource languages by using language-specific neurons. |
| Outcome: | The proposed method improves cross-lingual performance on low-resource languages by ensuring full activation of language overlap neurons. |
Similar Papers
From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing alignment benchmarks focus on sentence embeddings, but prior research has shown that neural models tend to induce a non-smooth representation space, which impact of semantic alignment evaluation on low-resource languages. |
| Approach: | They propose a novel cross-lingual alignment evaluation method based on the consistency of parallel sentences to assess model alignment. |
| Outcome: | The proposed method achieves a correlation of 0.9556 with downstream tasks performance and 0.8524 with transferability even with a small dataset. |
LLMs Are Few-Shot In-Context Low-Resource Language Learners (2024.naacl-long)
Copied to clipboard
| Challenge: | In-context learning (ICL) empowers large language models to perform diverse tasks in underrepresented languages using only short in-contrast information. |
| Approach: | They extensively assess the effectiveness of in-context learning with LLMs in low-resource languages . they also identify the shortcomings of in context label alignment . |
| Outcome: | The proposed approach improves understanding quality of low-resource languages by closing the language gap in the target language. |
Probing the Emergence of Cross-lingual Alignment during LLM Training (2024.findings-acl)
Copied to clipboard
| Challenge: | Multilingual Large Language Models (LLMs) achieve remarkable levels of zero-shot cross-lingual transfer performance. |
| Approach: | They propose that LLMs can align languages without explicit supervision from parallel sentences without a single linguistic feature. |
| Outcome: | The proposed model can perform zero-shot cross-lingual transfer even when the vocabularies of two languages have a null intersection, i.e., no tokens are shared. |
It’s All About In-Context Learning! Teaching Extremely Low-Resource Languages to LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Low-resource languages, especially those written in rare scripts, remain unsupported by large language models due to lack of training data. |
| Approach: | They evaluate 20 under-represented languages across three state-of-the-art multilingual LLMs and compare their methods to parameter-efficient fine-tuning. |
| Outcome: | The proposed methods compare with parameter-efficient fine-tuning (PEFT) on low-resource languages. |
Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Pre-trained multilingual models have shown great potential for zero-shot cross-lingual transfer to low web-resource languages (LRLs). |
| Approach: | They propose a vocabulary generation algorithm which enhances lexical overlap across related languages by generating a token that increases the representation of LRLs. |
| Outcome: | The proposed approach improves cross-lingual transfer accuracy without reducing HRL representation and accuracy. |
Low-resource Neural Machine Translation with Cross-modal Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural machine translation techniques rely on large monolingual corpus, which is costly for some low-resource languages. |
| Approach: | They propose a cross-modal contrastive learning method to learn a shared space for all languages by additional visual modality. |
| Outcome: | The proposed method can learn cross-modal and cross-lingual alignment with small amount of image-text pairs and achieves significant improvements over the text-only baseline. |
Neuron Specialization: Leveraging Intrinsic Task Modularity for Multilingual Machine Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Language-specific modeling methods that focus on heuristics for allocation of capacity and lack knowledge transfer capabilities are often prone to interference due to conflicting optimization demands. |
| Approach: | They propose a method that identifies specialized neurons to modularize feed-forward layers and updates them through sparse networks to avoid interference under multilingual translation. |
| Outcome: | The proposed approach achieves consistent performance gains over strong baselines with additional analyses showing reduced interference and increased knowledge transfer. |
Linguistic Minimal Pairs Elicit Linguistic Similarity in Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | a new analysis leverages linguistic minimal pairs to probe the internal linguistic representations of Large Language Models (LLMs). |
| Approach: | They propose to use linguistic minimal pairs to probe the internal linguistic representations of Large Language Models (LLMs). |
| Outcome: | The proposed analysis reveals that linguistic similarity is significantly influenced by training data exposure, leading to higher cross-LLM agreement in higher-resource languages. |
Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment (2023.acl-long)
Copied to clipboard
| Challenge: | a handful of studies have explored ICL in a cross-lingual setting . emergence of large-scale, pretrained, Transformer-based language models has marked the commencement of an avant-garde era in NLP. |
| Approach: | They propose a novel prompt construction strategy to bridge the gap between ICL and cross-lingual text classification. |
| Outcome: | The proposed approach outperforms random prompt selection by a large margin across three tasks using 44 different cross-lingual pairs. |
Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to address performance gaps in LLMs rely on pretraining or fine-tuning, which are resource-intensive. |
| Approach: | They propose a framework that aligns LLMs' internal representations with those of high-performing languages during inference. |
| Outcome: | The proposed framework improves performance on low-performing (source) languages by aligning their internal representations with those of high-performing languages during inference. |