Multilingual Brain Surgeon: Large Language Models Can Be Compressed Leaving No Language behind (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing methods for MC focus on quantization and network pruning. |
| Approach: | They propose a calibration method that samples calibration data from various languages proportionally to the language distribution of the model training datasets. |
| Outcome: | The proposed method improves the performance of existing English-centric compression methods on the BLOOM multilingual LLM. |
Similar Papers
LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent multilingual models support limited number of human languages due to lack of training data for low resource languages. |
| Approach: | They propose a multilingual multilingual LLM that scales to 100 languages . they use a human feedback dataset and a data set to perform multilingual instruction tuning . |
| Outcome: | The proposed model outperforms its peers on five multilingual benchmarks. |
VEEF-Multi-LLM: Effective Vocabulary Expansion and Parameter Efficient Finetuning Towards Multilingual Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a significant disadvantage for low-resource languages . VEEF-Multi-LLM-8B excels in multilingual instruction-following tasks . |
| Approach: | They propose a low-resource multilingual large language model that expands the vocabulary for multilingual support. |
| Outcome: | The proposed model outperforms existing models in multilingual instruction-following tasks, but lags behind English-centric models in some tasks. |
Everything you need to know about Multilingual LLMs: Towards fair, performant and reliable models for languages of the world (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Responsible AI issues such as fairness, bias and toxicity will be discussed in this tutorial . |
| Approach: | This tutorial will describe various aspects of scaling up language technologies to many of the world’s languages by describing the latest research in Massively Multilingual Language Models (MMLMs). |
| Outcome: | This tutorial will cover various aspects of scaling up language technologies to many of the world's languages by describing the latest research in multilingual models. |
Comparing Text Compression Capabilities of Large Language Models with Traditional Compression Algorithms (2026.eacl-srw)
Copied to clipboard
| Challenge: | Experimental results show that large language models outperform baselines on non-English datasets . traditional methods remained dataset-agnostic, and the results suggest that current methods are impractical for the compression task. |
| Approach: | They evaluate the non-English and unstructured text compression performance of Large Language Models . they compare them with traditional baselines on datasets from eight most widely spoken languages . |
| Outcome: | The evaluated LLM outperformed baselines on non-English datasets . the results show that the current methods are highly impractical for the compression task . |
Optimizing Language Augmentation for Multilingual Large Language Models: A Case Study on Korean (2024.lrec-main)
Copied to clipboard
ChangSu Choi, Yongbin Jeong, Seoyoon Park, Inho Won, HyeonSeok Lim, SangMin Kim, Yejee Kang, Chanhyuk Yoon, Jaewan Park, Yiseul Lee, HyeJin Lee, Younggyun Hahm, Hansaem Kim, KyungTae Lim
| Challenge: | Large language models (LLMs) use pretraining to predict the subsequent word, but less-resourced languages are being overlooked. |
| Approach: | They propose to expand the MLLM vocabularies to enhance expressiveness and use bilingual data for pretraining to align the high- and less-resourced languages. |
| Outcome: | The proposed model outperforms existing models in qualitative analyses compared to Korean monolingual models. |
Investigating the Multilingual Calibration Effects of Language Model Instruction Tuning (2026.eacl-short)
Copied to clipboard
Jerry Huang, Peng Lu, Qiuhao Zeng, Yusuke Iwasawa, Yutaka Matsuo, Sarath Chandar, Edison Marrese-Taylor, Irene Li
| Challenge: | despite advances in foundation model research, the relationship between large language models and their calibration remains an open area of research. |
| Approach: | They examine a gap in the calibration of large language models within multilingual settings to better understand how data scarcity can potentially lead to different calibration effects. |
| Outcome: | The proposed calibration gap is found in two multilingual benchmarks over 29 and 42 languages. |
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation? (2024.findings-eacl)
Copied to clipboard
Rishav Hada, Varun Gumma, Adrian Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, Sunayana Sitaram
| Challenge: | Large Language Models (LLMs) excel in various tasks, but their evaluation, especially in languages beyond the top 20, remains inadequate due to existing benchmarks and metrics limitations. |
| Approach: | They propose to use Large Language Models as evaluators to rank or score other models’ outputs by calibrating them against 20K human judgments across three text-generation tasks, five metrics, and eight languages. |
| Outcome: | The proposed evaluation methods can be used to improve multilingual evaluation by calibrating them against 20K human judgments across three text-generation tasks, five metrics, and eight languages. |
Data and Model Centric Approaches for Expansion of Large Language Models to New languages (2025.emnlp-tutorials)
Copied to clipboard
| Challenge: | Existing LLMs mainly support English alongside a handful of high resource languages . this leaves a major gap for most low-resource languages despite increasing pace of research . |
| Approach: | This tutorial examines approaches to expand the language coverage of LLMs . they look at tokenizer training, pre-training, instruction tuning, alignment, evaluation, etc. |
| Outcome: | This tutorial examines approaches to expand the language coverage of LLMs . it provides an efficient and viable path to bring LLM technologies to low-resource languages . |
Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Training (2024.findings-naacl)
Copied to clipboard
| Challenge: | Recent advances in language modeling have led to the emergence of large language models capable ofvarious natural language processing tasks. |
| Approach: | They propose a multi-instructional training approach that integrates a large language model with a speech encoder to harness the capabilities of LLMs for speech recognition and beyond. |
| Outcome: | The proposed model can be trained and aligned with a multilingual LLM on 1900 hours of transcribed data from 139 languages. |
X-LLaVA: Optimizing Bilingual Large Vision-Language Alignment (2024.findings-naacl)
Copied to clipboard
DongJae Shin, HyeonSeok Lim, Inho Won, ChangSu Choi, Minjun Kim, SeungWoo Song, HanGyeol Yoo, SangMin Kim, KyungTae Lim
| Challenge: | constructing multilingual data for large multimodal models presents its own set of challenges due to language diversity and complexity. |
| Approach: | They propose to use GPT4-V to construct multimodal training datasets using a text-only version of GPT4. |
| Outcome: | The proposed method performs well in Korean and English, surpassing existing methods. |