AdaMergeX: Cross-Lingual Transfer with Large Language Models via Adaptive Adapter Merging (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models excel in highresource languages but underperform in lowresource ones. |
| Approach: | They propose a cross-lingual transfer method that decouples "task ability" from "language ability" they propose to use adaptive adapter merging to obtain target adapters by combining other adapters. |
| Outcome: | The proposed method outperforms existing methods in highresource languages . it decouples "task ability" from "language ability" but fails to fully separate "task capability" from the "source language" |
Similar Papers
MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer (2020.emnlp-main)
Copied to clipboard
| Challenge: | Current deep pretrained models lack capacity to represent all languages . limited capacity is an issue even for high-resource languages where models are not included in training data at all. |
| Approach: | They propose an adapter-based framework that enables high portability and parameter-efficient transfer to arbitrary tasks and languages by learning modular language and task representations. |
| Outcome: | The proposed framework outperforms state-of-the-art models on cross-lingual transfer across languages and typologically diverse models. |
Cross-Lingual Optimization for Language Transfer in Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Adapting large language models to other languages often suffers from an overemphasis on English performance. |
| Approach: | They propose a cross-lingual optimization technique that efficiently transfers an English-centric LLM to a target language while preserving its English capabilities. |
| Outcome: | The proposed model outperforms SFT in acquiring target language proficiency and maintaining English performance in low-resource languages. |
MIGRATE: Cross-Lingual Adaptation of Domain-Specific LLMs through Code-Switching and Embedding Transfer (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have advanced in many fields, but focus on English-centric models requires extensive data. |
| Approach: | They propose a method that leverages open-source static embedding models and up to 3 million tokens of code-switching data to facilitate the seamless transfer of embeddables to target languages. |
| Outcome: | The proposed method outperforms baseline and existing cross-lingual transfer methods in target languages. |
Unifying Cross-Lingual Transfer across Scenarios of Resource Scarcity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to deal with resource scarcity have not been developed to deal effectively with the problem. |
| Approach: | They propose to use a set of tools to harness data from one or more high-resource "source" languages to compensate for a shortage of data in low-resourced "target" languages. |
| Outcome: | The proposed technique can be easily adapted to unseen languages, extending the range of the proposed technique and translation-based transfer more broadly. |
A Data-Efficient Path to Multilingual LLMs: Language Expansion via Post-training PARAM𝛥 Integration into Upcycled MoE (2026.acl-long)
Copied to clipboard
Hao Zhou, Tianhao Li, Zhijun Wang, Shuaijie She, Linjuan Wu, Hao-Ran Wei, Baosong Yang, Jiajun Chen, Shujian Huang
| Challenge: | Large Language Models (LLMs) are expensive and require extensive Continued Pre-Training and data-intensive alignment to expand. |
| Approach: | They propose a method which upcycles a dense model into a Mixture-of-Experts architecture, allocating different experts to different languages. |
| Outcome: | Experiments show that the proposed model upcycles a dense model into a Mixture-of-Experts(MoE) architecture, allocating different experts to different languages. |
Only for the Unseen Languages, Say the Llamas: On the Efficacy of Language Adapters for Cross-lingual Transfer in English-centric LLMs (2025.acl-srw)
Copied to clipboard
| Challenge: | Most state-of-the-art large language models (LLMs) are trained mainly on English data, limiting their effectiveness on non-English, especially low-resource, languages. |
| Approach: | They train language adapters for 13 languages and evaluate their effectiveness on downstream tasks using either task adapters or in-context learning. |
| Outcome: | The proposed language adapters improve performance for languages not seen during pretraining, but provide negligible benefit for seen languages. |
FAD-X: Fusing Adapters for Cross-lingual Transfer to Low-Resource Languages (2022.aacl-short)
Copied to clipboard
| Challenge: | Adapter-based tuning is a technique that selectively updates language-specific parameters to adapt to a new language, rather than fine-tuning all shared weights. |
| Approach: | They propose to add light-weight adapters to multilingual pretrained language models (mPLMs) and add language-specific parameters to adapt to a new language. |
| Outcome: | The proposed adapter can enhance cross-lingual transfer from pretrained adapters for well-known named entity recognition and classification benchmarks. |
Cross-Lingual Transfer with Target Language-Ready Task Adapters (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing frameworks for (zero-shot) cross-lingual transfer employ separate language and task adapters which can be arbitrarily combined to perform any task to any target language. |
| Approach: | They propose to fine-tune 'target language-ready' adapters to the target language to achieve better transfer performance without sacrificing the modularity of MAD-X. |
| Outcome: | The proposed adapters outperform MAD-X and BAD-X on most tasks and languages while maintaining the modularity of MAD. |
Phylogeny-Inspired Adaptation of Multilingual Models to New Languages (2022.aacl-main)
Copied to clipboard
| Challenge: | Large pretrained multilingual models have delivered promising results due to cross-lingual learning capabilities on a variety of language tasks. |
| Approach: | They propose to use language phylogenetic information to improve cross-lingual transfer by leveraging closely related languages in a structured, linguistically-informed manner. |
| Outcome: | The proposed model significantly improves on the baseline model on languages unseen during training. |
Extracting and Combining Abilities For Building Multi-lingual Ability-enhanced Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work relies on training with multi-lingual ability-related data, which may not be available for low-resource languages. |
| Approach: | They propose a multi-lingual ability-enhanced LLM that extracts language-agnostic ability-related weights from LLMs and combine them across different languages by simple addition and subtraction operations without training. |
| Outcome: | The proposed approach extracts language-agnostic ability-related weights from LLMs and combine them across different languages without training. |