When Life Gives You Samples: The Benefits of Scaling up Inference Compute for Multilingual LLMs (2025.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in large language models have shifted focus toward scaling inference-time compute. |
| Approach: | They propose to scale inference-time compute in a multilingual, multi-task setting . they propose to use m-ArenaHard-v2.0 prompts to sample multiple outputs in parallel . |
| Outcome: | The proposed solutions achieve an average +6.8 jump in win-rates for 8B models on m-ArenaHard-v2.0 prompts in non-English languages against proprietary models like Gemini. |
Similar Papers
Scaling LLM Inference Efficiently with Optimized Sample Compute Allocation (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to optimize sample allocations for large language models fail to account for the optimal sampling configuration. |
| Approach: | They propose an algorithm that optimizes sample allocation by finding an optimal mix of different inference configurations. |
| Outcome: | The proposed algorithm achieves better accuracy on SWE-Bench with 3x less compute than the default configuration. |
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized natural language processing and are limited by high inference time in multilingual settings. |
| Approach: | They propose a training recipe for an assistant model in speculative decoding, which are leveraged to draft and-then its future tokens are verified by the target LLM. |
| Outcome: | The proposed model significantly speeds up inference time and out-of-domain speedup across various languages. |
Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated multilingual capabilities, yet they are mostly English-centric due to the imbalanced training corpora. |
| Approach: | They extend the evaluation to real-world user queries and non-English-centric LLMs . they show that translation into English can boost LLM performance on NLP tasks . |
| Outcome: | The proposed evaluation extends to user queries and non-English-centric LLMs . it shows that translation into English can boost performance on NLP tasks, but not universally optimal . |
Breaking the Language Barrier: Can Direct Inference Outperform Pre-Translation in Multilingual LLM Applications? (2024.naacl-short)
Copied to clipboard
Yotam Intrator, Matan Halfon, Roman Goldenberg, Reut Tsarfaty, Matan Eyal, Ehud Rivlin, Yossi Matias, Natalia Aizenberg
| Challenge: | Existing studies have focused on pre-translation, but there is still need for it . authors say that it is not universally necessary to translate large language models . |
| Approach: | They re-evaluate the need for pre-translation in the context of PaLM2 models . authors found that PaLM2-L consistently outperforms pre-translated in 94 out of 108 languages . |
| Outcome: | The proposed model outperforms pre-translation in 94 out of 108 languages and 6 benchmarks . authors argue that pre-translated inputs can be used to improve performance . |
LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent multilingual models support limited number of human languages due to lack of training data for low resource languages. |
| Approach: | They propose a multilingual multilingual LLM that scales to 100 languages . they use a human feedback dataset and a data set to perform multilingual instruction tuning . |
| Outcome: | The proposed model outperforms its peers on five multilingual benchmarks. |
LLMs for Low Resource Languages in Multilingual, Multimodal and Dialectal Settings (2024.eacl-tutorials)
Copied to clipboard
| Challenge: | Recent advances in AI can be attributed to the remarkable performance of Large Language Models (LLMs) success of LLMs depends on specific training techniques, such as instruction tuning and prompting . |
| Approach: | They explore the capabilities of Large Language Models (LLMs) in various tasks and languages . they also examine their performance, fine-tuning, instructions tuning, and close vs. open models . |
| Outcome: | The proposed model can be used for speech and multimodal tasks across modalities, languages, and dialects. |
Large Language Models Are Bad Dice Players: LLMs Struggle to Generate Random Numbers from Statistical Distributions (2026.acl-long)
Copied to clipboard
| Challenge: | Existing large language models lack a functional internal sampler to faithfully sample from specified probability distributions . lack of robust sampling mechanisms across diverse application scenarios is a critical functional requirement . |
| Approach: | They propose to use a dual-protocol design to disentangle failure modes . batch generation achieves only modest statistical validity, while independent requests collapse almost entirely . |
| Outcome: | The proposed model fails to enforce uniform answer-position constraints and violates demographic targets in attribute-constrained text-to-image prompt synthesis. |
Current Advances in LLM Reasoning (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial examines comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) advanced inference time methods and post-training methods that aim to make LLMs think more like humans are discussed in this tutorial. |
| Approach: | This tutorial explores comprehensive evaluation strategies to assess the reasoning abilities of large language models (LLMs) and discusses two types of methods to improve models’ reasoning: advanced inference time methods, structured and self-improvement inference methods, and post-training methods, such as RLHF, DPO, and GRPO. |
| Outcome: | This tutorial examines evaluation strategies to assess the reasoning abilities of large language models and discusses two types of methods to improve models’ reasoning. |
The Best of Both Worlds: Combining Parallel and Sequential Inference Scaling via Aggregation Fine-Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show that AFT-trained models achieve substantial gains with test-time scaling. |
| Approach: | They introduce a supervised fine-tuning paradigm where models synthesize multiple draft responses into a single, refined answer. |
| Outcome: | Empirical results show that AFT-trained models outperform baseline models while eliminating external guidance. |
Linguistic Generalizability of Test-Time Scaling in Mathematical Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that pre-training compute can improve multilingual performance, but is it effective for test-time scaling? |
| Approach: | They propose a multilingual math benchmark with competition-level problems in 55 languages . they propose "test-time scaling" which further lengthens the time it takes to scale . |
| Outcome: | The proposed methods fail to generalize robustly across languages, with no improvements in variance or consistency. |