Papers with LLaMAX
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit exceptional translation capabilities in high-resource language tasks, yet their effectiveness in low-resourced languages is suboptimal. |
| Approach: | They conduct extensive multilingual continual pre-training on the LLaMA series models and develop LLiMAX for translation support across more than 100 languages. |
| Outcome: | The proposed model achieves higher translation performance than existing open-source models and performs on-par with specialized translation model on the Flores-101 benchmark. |
Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training large language models rely heavily on high-quality parallel data, which are often scarce or unavailable for low-resource languages. |
| Approach: | They propose a reinforcement training method using only monolingual text to elevate LLMs’ translation capabilities on massive low-resource languages while retaining their performance on high-resourced languages. |
| Outcome: | The proposed model outperforms LLaMAX, one of the strongest open-source multilingual LLMs on 1,414 language directions on Flores-101 dataset. |