| Challenge: | We propose a new multilingual language model benchmark that is composed of 40+ languages spanning several scripts and linguistic families. |
| Approach: | They propose a multilingual language model benchmark composed of 40+ languages . they train monolingual causal language models using a state-of-the-art model . |
| Outcome: | The proposed model is composed of 40+ languages spanning several scripts and linguistic families. |
Similar Papers
BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing multilingual benchmarks focus primarily on language understanding tasks. |
| Approach: | They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages. |
| Outcome: | Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve. |
When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual language models are widely used to extend NLP systems to low-resource languages. |
| Approach: | They pre-train over 10,000 monolingual and multilingual language models for over 250 languages including multiple language families that are under-studied in NLP. |
| Outcome: | The results show that adding multilingual data improves low-resource language modeling performance, similar to increasing low-source dataset sizes by up to 33%. |
Models and Datasets for Cross-Lingual Summarisation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent years have witnessed increased interest in abstractive summarisation thanks to the popularity of neural network models and the availability of datasets containing hundreds of thousands of document-summary pairs. |
| Approach: | They propose to create a cross-lingual summarisation corpus with long documents in a source language associated with multi-sentence summaries in . target language. |
| Outcome: | The proposed task can be applied to several other languages and covers twelve languages and directions. |
MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairs (2026.tacl-1)
Copied to clipboard
| Challenge: | MultiBLiMP 1.0 is a massively multilingual benchmark of linguistic minimal pairs covering 101 languages and 2 types of subject-verb agreement. |
| Approach: | They propose to use multilingual benchmarks to evaluate linguistic minimal pairs in 101 languages and 2 types of subject-verb agreement to create the minimal pairs. |
| Outcome: | The proposed benchmark covers 101 languages and 2 types of subject-verb agreement, and contains more than 128,000 minimal pairs. |
Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages (2023.acl-long)
Copied to clipboard
Ayyoob ImaniGooghari, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Martins, François Yvon, Hinrich Schütze
| Challenge: | Lack of LLMs supporting low-resource languages is a serious impediment to bringing NLP to all of the world. |
| Approach: | They create a model that scales LLMs horizontally and a corpus that covers 511 low-resource languages. |
| Outcome: | The proposed model improves on five diverse tasks across low- and high-resource languages. |
SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects (2024.eacl-long)
Copied to clipboard
David Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba Alabi, Yanke Mao, Haonan Gao, En-Shiun Lee
| Challenge: | despite progress in building multilingual language models evaluation is limited to a few languages with available datasets . despite this, we create a large-scale open-sourced benchmark dataset for topic classification in 205 languages and dialects to address the lack of evaluation dataset for Natural Language Understanding (NLU). |
| Approach: | They create a large-scale open-sourced benchmark dataset for topic classification in 205 languages and dialects to address the lack of evaluation dataset for Natural Language Understanding (NLU). |
| Outcome: | The proposed dataset addresses the lack of evaluation dataset for Natural Language Understanding (NLU) for many languages, it is the first publicly available evaluation dataset. |
A Recipe of Parallel Corpora Exploitation for Multilingual Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have highlighted the potential of exploiting parallel corpora to enhance multilingual large language models. |
| Approach: | They investigate the impact of parallel corpora quality and quantity, training objectives, and model size on performance of multilingual large language models enhanced with parallel corporeal. |
| Outcome: | The proposed approach improves performance in bilingual and general-purpose tasks. |
FuxiTranyu: A Multilingual Large Language Model Trained with Balanced Data (2024.emnlp-industry)
Copied to clipboard
Haoran Sun, Renren Jin, Shaoyang Xu, Leiyu Pan, null Supryadi, Menglong Cui, Jiangcun Du, Yikun Lei, Lei Yang, Ling Shi, Juesi Xiao, Shaolin Zhu, Deyi Xiong
| Challenge: | Large language models exhibit significant performance discrepancies between high- and low-resource languages. |
| Approach: | They present an open-source multilingual LLM with 8 billion parameters and a multilingual instruction dataset. |
| Outcome: | The proposed model achieves consistent multilingual representations across languages. |
Taxi1500: A Dataset for Multilingual Text Classification in 1500 Languages (2025.naacl-short)
Copied to clipboard
| Challenge: | a large-scale text classification dataset encompassing 1504 languages is needed to address this gap . low-resource languages are often overlooked due to the scarcity of evaluation datasets. |
| Approach: | They propose to use translations of the Bible to construct a large-scale text classification dataset that covers 1504 languages and annotate them using crowdsourcing. |
| Outcome: | The proposed dataset covers 1504 languages and is available to the public. |
An Analysis of Massively Multilingual Neural Machine Translation for Low-Resource Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | In this study, we explore massively multilingual low-resource neural machine translation. |
| Approach: | They propose to use Bible translations to train models with up to 1,107 source languages and create multilingual corpora varying the number and relatedness of source languages. |
| Outcome: | The proposed approach is highly language-specific and can be tailored to the source language and its typology. |