Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive translation capabilities even without being explicitly trained on parallel data. |
| Approach: | They propose to add parallel data to enhance multilingual encoder-based and encoder decoder language models by focusing on translation and multilingual common-sense reasoning. |
| Outcome: | The proposed methods show that adding parallel data can significantly improve LLMs’ multilingual capabilities. |
Similar Papers
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |
Revealing the Parallel Multilingual Learning within Large Language Models (2024.emnlp-main)
Copied to clipboard
Yongyu Mu, Peinan Feng, Zhiquan Cao, Yuzhang Wu, Bei Li, Chenglong Wang, Tong Xiao, Kai Song, Tongran Liu, Chunliang Zhang, JingBo Zhu
| Challenge: | Large language models (LLMs) can handle multilingual and cross-lingual text within a single input; however, previous studies focusing on using English as the pivot language to enhance language understanding and reasoning focus on using multiple languages. |
| Approach: | They propose to use parallel multilingual input to enhance the model's comprehension of the input and to examine how multilingual processing affects prediction. |
| Outcome: | The proposed model can handle multilingual and cross-lingual text within a single input, but previous studies focused on using English as the pivot language to enhance language understanding and reasoning. |
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis (2024.findings-naacl)
Copied to clipboard
Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li
| Challenge: | Existing studies show that large language models (LLMs) can handle multilingual machine translation (MMT) However, the multilingual translation ability of LLMs remains under-explored. |
| Approach: | They evaluate eight popular LLMs including ChatGPT and GPT-4 to determine their performance in multilingual machine translation. |
| Outcome: | The proposed model can generate moderate translation even on zero-resource languages and cross-lingual exemplars can provide better task guidance for low-resourced translation than exemplar in the same language pairs. |
Fine-Tuning Large Language Models to Translate: Will a Touch of Noisy Data in Misaligned Languages Suffice? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Traditionally, success in multilingual machine translation depends on large volume, diverse directions, and high quality of training data. |
| Approach: | They revisit the importance of large language models for translation by fine-tuning on 32 parallel sentences. |
| Outcome: | The proposed model can be fine-tuned on as few as 32 parallel sentences . however, the choice of direction is critical to avoid misinterpretation, the authors say . |
Towards Practical and Knowledgeable LLMs for a Multilingual World: A Thesis Proposal (2025.naacl-srw)
Copied to clipboard
| Challenge: | a proposed thesis examines the role that multilinguality occupies in the development of practical and knowledgeable LLMs. |
| Approach: | They propose to use multilingual knowledge to improve LLM performance on NLP tasks . they extend the territorial disputes benchmark to retrieval-augmented generation setting . |
| Outcome: | The proposed methods improve LLMs' performance on standard natural language processing tasks by leveraging their existing multilingual knowledge. |
How Vocabulary Sharing Facilitates Multilingualism in LLaMA? (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) show strong performance on English tasks, but their performance in other languages is limited. |
| Approach: | They conducted an exhaustive analysis of the multilingual capability of LLMs by examining the performance gap before and after embedding fine-tuning across 101 languages. |
| Outcome: | The proposed model improves on the attributes of four quadrants in the model and provides actionable and efficient guidelines for tuning these languages. |
On the Role of Parallel Data in Cross-lingual Transfer Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing multilingual models do not exploit the full potential of monolingual data, a new study finds . prior work has shown that parallel data is beneficial for cross-lingual learning, but it is unclear if it is the data itself or the modeling of parallel interactions that matters. |
| Approach: | They compare unsupervised machine translation to supervised machine translator and gold parallel data to generate synthetic parallel data. |
| Outcome: | The proposed model generated parallel data is better than supervised machine translation and gold parallel data in both general and task-specific settings. |
Parallel Universes, Parallel Languages: A Comprehensive Study on LLM-based Multilingual Counterfactual Example Generation (2026.acl-long)
Copied to clipboard
Qianli Wang, Van Bach Nguyen, Yihong Liu, Fedor Splitt, Nils Feldhus, Christin Seifert, Hinrich Schuetze, Sebastian Möller, Vera Schmitt
| Challenge: | Large language models excel at generating English counterfactuals but their effectiveness in generating multilingual counterfacts remains unclear. |
| Approach: | They conduct automatic evaluations on both directly generated and derived counterfactuals in six languages and find that cross-lingual perturbations follow common strategic principles. |
| Outcome: | The proposed models show that translation-based counterfactuals offer higher validity than their directly generated counterparts, but still fall short of matching the quality of the original English counterf actuals. |
Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have shown continuously improving multilingual capabilities. |
| Approach: | They evaluate the ability of open LLMs to handle multilingual machine translation tasks using a parallel-first monolingual-second data mixing strategy. |
| Outcome: | The proposed model outperforms state-of-the-art models and achieves competitive performance with Google Translate and GPT-4-turbo. |