Representational Isomorphism and Alignment of Multilingual Large Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing isomorphism of sentence representations can facilitate representational alignments in zero-shot and few-shot settings. |
| Approach: | They propose to apply a contrastive objective to LLMs with a small number of translation pairs to improve models' performance on Semantic Textual Similarity tasks. |
| Outcome: | The proposed representation-level approach significantly improves on Semantic Textual Similarity (STS) tasks across languages even without a monolingual objective. |
Similar Papers
AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities, but performance and cross-lingual alignment often lag for non-dominant languages. |
| Approach: | They propose a representation-level framework to enhance multilingual performance of pre-trained LLMs by integrating multilingual semantic alignment and language feature integration. |
| Outcome: | The proposed framework improves multilingual capability of pre-trained LLMs by bringing representations closer and improving cross-lingual alignment. |
Can you map it to English? The Role of Cross-Lingual Alignment in the Multilingual Performance of LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can answer prompts in many languages despite being pre-trained mostly on English text. |
| Approach: | They propose a Discriminative Alignment Index to quantify instance-level alignment across 24 languages other than English and three distinct NLU tasks. |
| Outcome: | The proposed model can perform natural language understanding tasks in 24 languages other than English and three distinct NLU tasks. |
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
Concept Space Alignment in Multilingual LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large language models generalize somewhat across languages, but it is unclear whether this is a result of improved, implicit alignment, or of something else, e.g., linguistic overlap or semi-parallel subsets of training data. |
| Approach: | They hypothesize that implicit alignment is the reason for generalization in multilingual large language models. |
| Outcome: | The proposed model generalizes well across languages, but lacks linearity. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |
From Neurons to Semantics: Evaluating Cross-Linguistic Alignment Capabilities of Large Language Models via Neurons Alignment (2025.acl-long)
Copied to clipboard
| Challenge: | Existing alignment benchmarks focus on sentence embeddings, but prior research has shown that neural models tend to induce a non-smooth representation space, which impact of semantic alignment evaluation on low-resource languages. |
| Approach: | They propose a novel cross-lingual alignment evaluation method based on the consistency of parallel sentences to assess model alignment. |
| Outcome: | The proposed method achieves a correlation of 0.9556 with downstream tasks performance and 0.8524 with transferability even with a small dataset. |
Probing the Emergence of Cross-lingual Alignment during LLM Training (2024.findings-acl)
Copied to clipboard
| Challenge: | Multilingual Large Language Models (LLMs) achieve remarkable levels of zero-shot cross-lingual transfer performance. |
| Approach: | They propose that LLMs can align languages without explicit supervision from parallel sentences without a single linguistic feature. |
| Outcome: | The proposed model can perform zero-shot cross-lingual transfer even when the vocabularies of two languages have a null intersection, i.e., no tokens are shared. |
Improving In-context Learning of Multilingual Generative Language Models with Cross-lingual Alignment (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing studies show that multilingual generative models exhibit a strong language bias toward high-resource languages. |
| Approach: | They propose a cross-lingual alignment framework exploiting pairs of translation sentences to improve cross-linguistic abilities. |
| Outcome: | The proposed framework improves cross-lingual abilities and mitigates performance gap. |
Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Effective cross-lingual transfer is hindered by performance gaps and the scarcity of fine-tuning data in many languages. |
| Approach: | They propose a middle-layer alignment objective integrated into task-specific training to improve cross-lingual transfer across languages. |
| Outcome: | The proposed method improves cross-lingual transfer to lower-resource languages and can be merged with existing modules without full re-training. |