FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data (2024.emnlp-industry)
Copied to clipboard
Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, null Supryadi, Menglong Cui, Jiangcun Du, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, Shaolin Zhu, Deyi Xiong
| Challenge: | Large language models exhibit significant performance discrepancies between high- and low-resource languages. |
| Approach: | They present an open-source multilingual LLM with 8 billion parameters and a multilingual instruction dataset. |
| Outcome: | The proposed model achieves consistent multilingual representations across languages. |
Similar Papers
VEEF-Multi-LLM: Effective Vocabulary Expansion and Parameter Efficient Finetuning Towards Multilingual Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a significant disadvantage for low-resource languages . VEEF-Multi-LLM-8B excels in multilingual instruction-following tasks . |
| Approach: | They propose a low-resource multilingual large language model that expands the vocabulary for multilingual support. |
| Outcome: | The proposed model outperforms existing models in multilingual instruction-following tasks, but lags behind English-centric models in some tasks. |
RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Preference optimization is a widely adopted post-training technique to align large language models with human preferences. |
| Approach: | They propose a method for generating multilingual feedback data to balance data coverage. |
| Outcome: | The proposed method achieves 54.4% win-rate against current state-of-the-art multilingual LLM in its parameter class and 69.5% win- rate or higher against widely used models like Gemma, Mistral and Llama 3. |
Paramanu: Compact and Competitive Monolingual Language Models for Low-Resource Morphologically Rich Indian Languages (2026.acl-long)
Copied to clipboard
| Challenge: | Multilingual large language models are expensive to pretrain and suffer from imbalances across languages and datasets. |
| Approach: | They propose a family of Indian language-only autoregressive language models trained on open-source language-specific data for the five most spoken Indian languages. |
| Outcome: | The proposed model outperforms most larger models up to 8B across all five languages. |
Turning English-centric LLMs Into Polyglots: How Much Multilinguality Is Needed? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models that target a single language are not seen during finetuning, but are able to respond in multiple languages once deployed in downstream applications. |
| Approach: | They investigate the minimal amount of multilinguality required during finetuning to elicit effective cross-lingual generalisation in English-centric LLMs. |
| Outcome: | The proposed model can respond in as few as two to three languages to a user's query in English, but the degree to which a target language is seen during pretraining is limiting. |
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks focus primarily on language understanding tasks. |
| Approach: | They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages. |
| Outcome: | Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve. |
Cool-Fusion: Fuse Large Language Models without Training (2025.acl-long)
Copied to clipboard
| Challenge: | Cool-Fusion is a simple yet effective approach to combine two or more heterogeneous large language models . |
| Approach: | They propose a method that fuses the knowledge of two or more heterogeneous large language models to leverage complementary strengths. |
| Outcome: | The proposed method increases accuracy from three strong source LLMs on GSM8K by 17.4%. |
LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent multilingual models support limited number of human languages due to lack of training data for low resource languages. |
| Approach: | They propose a multilingual multilingual LLM that scales to 100 languages . they use a human feedback dataset and a data set to perform multilingual instruction tuning . |
| Outcome: | The proposed model outperforms its peers on five multilingual benchmarks. |
Getting More from Less: Large Language Models are Good Spontaneous Multilingual Learners (2024.emnlp-main)
Copied to clipboard
Shimao Zhang, Changjiang Gao, Wenhao Zhu, Jiajun Chen, Xin Huang, Xue Han, Junlan Feng, Chao Deng, Shujian Huang
| Challenge: | Large Language Models (LLMs) have shown impressive language capabilities, but most of them have very unbalanced performance across different languages. |
| Approach: | They propose to use question translation data to enhance LLMs' multilingual capabilities by using mechanistic interpretability methods. |
| Outcome: | The proposed method improves multilingual alignment even with unannotated answers in English and a wide range of languages even with instruction-tuned LLMs. |
CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages (2024.lrec-main)
Copied to clipboard
Thuat Nguyen, Chien Van Nguyen, Viet Dac Lai, Hieu Man, Nghia Trung Ngo, Franck Dernoncourt, Ryan A. Rossi, Thien Huu Nguyen
| Challenge: | Existing training datasets for large language models are often not fully disclosed. |
| Approach: | They propose a multilingual dataset with 6.3 trillion tokens in 167 languages . they use a pipeline of multiple stages to achieve the best quality for model training . |
| Outcome: | The proposed dataset is cleaned and deduplicated to achieve the best quality for model training . lack of transparency has hindered research on attributing and addressing hallucination and bias issues . 6.3 trillion tokens in 167 languages are used to train multilingual LLMs . |
Okapi: Instruction-tuned Large Language Models in Multiple Languages with Reinforcement Learning from Human Feedback (2023.emnlp-demo)
Copied to clipboard
| Challenge: | Existing instruction-tuned open-source LLMs have only been instruction- tuned for English and a few popular languages, thus hindering their accessibility to many other languages in the world. |
| Approach: | They propose a framework that uses supervised fine-tuning and reinforcement learning from human feedback to improve the accessibility of large language models. |
| Outcome: | The proposed framework enables the evaluation of generative LLMs in multiple languages. |