Bridging the Language Gaps in Large Language Models with Inference-Time Cross-Lingual Intervention (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to address performance gaps in LLMs rely on pretraining or fine-tuning, which are resource-intensive. |
| Approach: | They propose a framework that aligns LLMs' internal representations with those of high-performing languages during inference. |
| Outcome: | The proposed framework improves performance on low-performing (source) languages by aligning their internal representations with those of high-performing languages during inference. |
Similar Papers
An Empirical Study on Cross-lingual Vocabulary Adaptation for Efficient Language Model Inference (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-lingual vocabulary adaptation (CVA) methods have been proposed for adapting models to a target language . but effectiveness of these methods on increasing inference efficiency of generative large language models has not been explored. |
| Approach: | They propose to use cross-lingual vocabulary adaptation methods to adapt models to a target language to improve downstream performance. |
| Outcome: | The proposed methods significantly speed up models in four languages and four natural language understanding tasks. |
Word-level Cross-lingual Structure in Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated exceptional performance across a broad spectrum of cross-lingual Natural Language Processing (NLP) tasks. |
| Approach: | They propose to use Word-level Cross-lingual Structure to prove that the word-level embedding on the hidden layers isomorphic between languages. |
| Outcome: | The proposed method significantly improves on two representative LLM foundations, LLaMA2 and BLOOM. |
Less is More: Improving LLM Reasoning with Minimal Test-Time Intervention (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have focused on test-time scaling to improve reasoning quality but at the cost of efficiency. |
| Approach: | They propose a training-free framework that enhances reasoning accuracy and stability with minimal overhead. |
| Outcome: | The proposed framework yields consistent gains across general, coding, and STEM tasks while remaining highly efficient. |
Unlocking Multilingual Reasoning Capability of LLMs and LVLMs through Representation Engineering (2026.acl-long)
Copied to clipboard
Qiming Li, Xiaocheng Feng, Yixuan Ma, Ruihan Chen, Zihe Tong, Zekai Ye, Xiachong Feng, Libo Qin, Haoyu Ren, Kun Chen, Yunfei Lu, Dandan Tu, Bing Qin
| Challenge: | Existing approaches to enhance multilingual reasoning capabilities rely on costly multilingual training or employ prompting with external translation tools. |
| Approach: | They propose a training-free inference-time method to enhance multilingual reasoning capabilities via Representation Engineering without additional training data or tools. |
| Outcome: | The proposed method outperforms existing methods on four reasoning benchmarks in English and Thai and Swahili. |
GrAInS: Gradient-based Attribution for Inference-Time Steering of LLMs and VLMs (2026.acl-long)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning large language models often ignore token-level causal influence and underutilize model logits. |
| Approach: | They propose a novel approach that uses a gradient-based approach to identify influential tokens and construct directional steering vectors based on their contribution to preferred over dispreferred outputs. |
| Outcome: | The proposed approach outperforms fine-tuning and prior steering methods on both LLM and VLM tasks without degrading fluency or general capabilities. |
Middle-Layer Representation Alignment for Cross-Lingual Transfer in Fine-Tuned LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Effective cross-lingual transfer is hindered by performance gaps and the scarcity of fine-tuning data in many languages. |
| Approach: | They propose a middle-layer alignment objective integrated into task-specific training to improve cross-lingual transfer across languages. |
| Outcome: | The proposed method improves cross-lingual transfer to lower-resource languages and can be merged with existing modules without full re-training. |
AlignX: Advancing Multilingual Large Language Models with Multilingual Representation Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Multilingual large language models (LLMs) possess impressive multilingual understanding and generation capabilities, but performance and cross-lingual alignment often lag for non-dominant languages. |
| Approach: | They propose a representation-level framework to enhance multilingual performance of pre-trained LLMs by integrating multilingual semantic alignment and language feature integration. |
| Outcome: | The proposed framework improves multilingual capability of pre-trained LLMs by bringing representations closer and improving cross-lingual alignment. |
Mitigating Language Confusion through Inference-time Intervention (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to address the problem of language confusion are incontext learning and supervised fine-tuning (SFT) however, they consume context window space and require extensive data collection. |
| Approach: | They propose a language-sensitive intervention that detects and assesses language confusion without additional complex mechanisms. |
| Outcome: | The proposed method detects language confusion and assesses content quality without additional complex mechanisms. |
Can you map it to English? The Role of Cross-Lingual Alignment in the Multilingual Performance of LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can answer prompts in many languages despite being pre-trained mostly on English text. |
| Approach: | They propose a Discriminative Alignment Index to quantify instance-level alignment across 24 languages other than English and three distinct NLU tasks. |
| Outcome: | The proposed model can perform natural language understanding tasks in 24 languages other than English and three distinct NLU tasks. |
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized natural language processing and are limited by high inference time in multilingual settings. |
| Approach: | They propose a training recipe for an assistant model in speculative decoding, which are leveraged to draft and-then its future tokens are verified by the target LLM. |
| Outcome: | The proposed model significantly speeds up inference time and out-of-domain speedup across various languages. |