Error Analysis of Multilingual Language Models in Machine Translation: A Case Study of English-Amharic Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large language models have significantly advanced machine translation, yet challenges remain for low-resource languages like Amharic. |
| Approach: | They evaluated the performance of NLLB-200 and M2M in English-Amharic bidirectional translation using the Lesan AI dataset. |
| Outcome: | The proposed models outperformed the existing models in English-Amharic bidirectional translation using the Lesan AI dataset. |
Similar Papers
Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis (2024.findings-naacl)
Copied to clipboard
Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, Lei Li
| Challenge: | Existing studies show that large language models (LLMs) can handle multilingual machine translation (MMT) However, the multilingual translation ability of LLMs remains under-explored. |
| Approach: | They evaluate eight popular LLMs including ChatGPT and GPT-4 to determine their performance in multilingual machine translation. |
| Outcome: | The proposed model can generate moderate translation even on zero-resource languages and cross-lingual exemplars can provide better task guidance for low-resourced translation than exemplar in the same language pairs. |
Explainability and Interpretability of Multilingual Large Language Models: A Survey (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing literature on multilingual large language models lacks transparency in their internal processes. |
| Approach: | They propose to use multilingual large language models to examine their explainability and interpretability methods. |
| Outcome: | The present study examines the explainability and interpretability of multilingual large language models. |
On the Calibration of Massively Multilingual Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Massively Multilingual Language Models (MMLMs) have gained popularity due to their effectiveness in cross-lingual transfer. |
| Approach: | They investigate how well calibrated MMLMs are with respect to confidence . they find that calibration methods like temperature scaling and label smoothing improve calibration . |
| Outcome: | The proposed models are able to generalize in languages unseen during fine-tuning, but they are not reliable across languages. |
Everything you need to know about Multilingual LLMs: Towards fair, performant and reliable models for languages of the world (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Responsible AI issues such as fairness, bias and toxicity will be discussed in this tutorial . |
| Approach: | This tutorial will describe various aspects of scaling up language technologies to many of the world’s languages by describing the latest research in Massively Multilingual Language Models (MMLMs). |
| Outcome: | This tutorial will cover various aspects of scaling up language technologies to many of the world's languages by describing the latest research in multilingual models. |
What do Large Language Models Need for Machine Translation Evaluation? (2024.emnlp-main)
Copied to clipboard
Shenbin Qian, Archchana Sindhujan, Minnie Kabra, Diptesh Kanojia, Constantin Orasan, Tharindu Ranasinghe, Fred Blain
| Challenge: | Existing research shows that large language models can perform better in machine translation tasks. |
| Approach: | They propose to use large language models for machine translation evaluations . authors explore what translation information is needed for LLMs to evaluate MT quality . |
| Outcome: | The proposed model performs comparable to fine-tuned multilingual pre-trained models. |
To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages (2024.naacl-long)
Copied to clipboard
| Challenge: | XLT with multilingual language models is superfluous, says a new study . mBERT, XLM-R and mT5 are effective for cross-lingual transfer, authors say . |
| Approach: | They propose to use multilingual language models to improve cross-lingual transfer (XLT) they propose to add reliable translations to training data for XLT even for non-MT languages . |
| Outcome: | The proposed approaches outperform zero-shot XLT with mLMs, the authors show . the authors believe their findings warrant a broader inclusion of more robust translation-based baselines in XL research. |
Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation (2026.acl-long)
Copied to clipboard
| Challenge: | Machine-translated benchmarks are widely used to assess the multilingual capabilities of large language models (LLMs), yet translation errors in these benchmarks remain underexplored. |
| Approach: | They show how well machine-translated benchmarks match human span annotations on translations . they also show how strongly translation errors explain accuracy drops on translated benchmarks - a gap that is not addressed yet . |
| Outcome: | The proposed model matches human-level translations with human-language annotations on translations, but translation errors are associated with accuracy drops even after controlling for English correctness and source-side anomalies. |
Lost in the Source Language: How Large Language Models Evaluate the Quality of Machine Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that Large Language Models (LLMs) can be used as translation evaluators. |
| Approach: | They propose to use both coarse-grained and fine-grounded prompts to discern the utility of source versus reference data in machine translation evaluation tasks. |
| Outcome: | The proposed model can be used to evaluate translations in multiple languages. |
Many-to-English Machine Translation Tools, Data, and Pretrained Models (2021.acl-demo)
Copied to clipboard
| Challenge: | Commercial translation systems support only one hundred languages or fewer . commercial translation systems do not make these models available for transfer to low resource languages . |
| Approach: | They propose a multilingual neural machine translation model that can translate from 500 source languages to English. |
| Outcome: | The proposed model can translate from 500 source languages to English, or be used as a parent model for low-resource languages. |
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained Multilingual Language Models have shown a strong ability to transfer knowledge across languages. |
| Approach: | They examine factors contributing to the ability of MLLMs to perform zero-shot cross-lingual transfer . they identify consensuses among studies with consistent findings and resolve conflicts . |
| Outcome: | The authors outline and discuss factors that contribute to the ability of MLLMs to perform zero-shot cross-lingual transfer. |