A Fair Comparison without Translationese: English vs. Target-language Instructions for Multilingual LLMs (2025.naacl-short)
Copied to clipboard
| Challenge: | Prior studies suggested that English instructions are more effective for non-English tasks . however, these studies often use datasets and instructions translated from English . |
| Approach: | They conduct a fair comparison between English and target-language instructions by eliminating translationese effects. |
| Outcome: | The results show that the advantage of adopting English instructions is not overwhelming . the results also show that instruction-following abilities are improved when using respective instructions. |
Similar Papers
Beyond English: The Impact of Prompt Translation Strategies across Languages and Tasks in Multilingual LLMs (2025.findings-naacl)
Copied to clipboard
| Challenge: | Current LLMs are primarily trained on English data but also include data from other languages. |
| Approach: | They propose to use a pre-translation strategy to translate a task prompt into English before inference . they use 'a modular entity' that could be translated into four different languages . |
| Outcome: | The proposed strategies are based on a set of pre-trained data across 35 languages covering both low and high-resource languages. |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |
Ready to Translate, Not to Represent? Bias and Performance Gaps in Multilingual LLMs Across Language Families and Domains (2026.findings-acl)
Copied to clipboard
Md. Faiyaz Abdullah Sayeedi, Subhey Sadi Rahman, Md. Mahbub Alam, Md. Adnanul Islam, Jannatul Ferdous Deepti, Tasnim Mohiuddin, Md Mofijul Islam, Swakkhar Shatabda
| Challenge: | Large Language Models (LLMs) have redefined Machine Translation, enabling context-aware and fluent translations across hundreds of languages and textual domains. |
| Approach: | They propose a framework and dataset to evaluate the translation quality and fairness of open-source LLMs. |
| Outcome: | The proposed framework and dataset evaluates translation quality and fairness of open-source LLMs. |
Safety of Large Language Models Beyond English: A Systematic Literature Review of Risks, Biases, and Safeguards (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have a growing number of applications that generate harmful, biased, or unsafe content. |
| Approach: | They synthesize findings from recent studies that evaluate their robustness across languages . they highlight gaps in multilingual safety research and recommend future work . |
| Outcome: | The systematic review examines the multilingual safety of large language models in English . it identifies challenges such as dataset availability and evaluation biases . |
Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing practices of fine-tuning and evaluating multilingual large language models may not align with this objective due to a heavy reliance on translation. |
| Approach: | They propose to use translated or native instruction data to fine-tune multilingual large language models. |
| Outcome: | The proposed model can be fine tuned and evaluated in multilingual large language models . the results show that native or translated data can be used to compare model performance . |
Do Large Language Models have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Current Large Language Models (LLMs) are predominantly designed with English as the primary language, but many are still English-dominated. |
| Approach: | They propose to use automatic corpus-level metrics to assess lexical and syntactic naturalness of LLMs in a multilingual context. |
| Outcome: | The proposed method improves naturalness of LLMs in target languages without compromising performance on general-purpose benchmarks. |
7 Points to Tsinghua but 10 Points to ? Assessing Large Language Models in Agentic Multilingual National Bias (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models have garnered significant attention for their capabilities in multilingual natural language processing, but studies on risks associated with cross biases are limited to immediate context preferences. |
| Approach: | They investigate multilingual bias in state-of-the-art Large Language Models by analyzing their responses to decision-making tasks across multiple languages. |
| Outcome: | The proposed model can provide personalized advice across university applications, travel, and relocation scenarios. |
EXECUTE: A Multilingual Benchmark for LLM Token Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | EXECUTE is an expandable X(Cross)-Lingual Extension of CUTE that can be expanded to any language. |
| Approach: | They extend the CUTE benchmark to more languages with diverse scripts and writing systems, introducing EXECUTE. |
| Outcome: | The extended framework allows expansion to any language. |
Investigating Multilingual Instruction-Tuning: Do Polyglot Models Demand for Multilingual Instructions? (2024.emnlp-main)
Copied to clipboard
Alexander Weber, Klaudia Thellmann, Jan Ebert, Nicolas Flores-Herr, Jens Lehmann, Michael Fromm, Mehdi Ali
| Challenge: | a study of multilingual pre-trained LLMs on parallel instruction-tuning benchmarks shows that instruction-following models can be used across languages by up to 9.9%. |
| Approach: | They conduct an extensive study of the performance of multilingual pre-trained LLMs instruction-tuned on parallel instruction-uning datasets. |
| Outcome: | The proposed model improves cross-lingual instruction following capabilities by 9.9% on a large and mid-sized LLM on parallel instruction-tuning datasets. |
How Does Quantization Affect Multilingual LLMs? (2024.findings-emnlp)
Copied to clipboard
Kelly Marchisio, Saurabh Dash, Hongyu Chen, Dennis Aumiller, Ahmet Üstün, Sara Hooker, Sebastian Ruder
| Challenge: | Quantization is widely used to improve inference speed and deployment of large language models. |
| Approach: | They conduct a thorough analysis of quantized multilingual LLMs . they find language disparately affected by quantization, non-Latin script languages worst . authors urge consideration of multilingual performance as evaluation criterion for efficient models . |
| Outcome: | The results show that quantization has harmful effects on human evaluation . language performance is disparately affected by quantization, the authors say . |