Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that multilingual language models (MultiLMs) are capable of logically reasoning over natural language statements, reasoning with their implicit knowledge, and performing multi-step reasoning when the model size is large enough. |
| Approach: | They propose a mechanism that encourages cross-lingual attention in code-switched sequences and improves reasoning performance by up to 14%. |
| Outcome: | The proposed approach improves reasoning performance by 14% and 4% on the RuleTaker and LeapOfThought datasets. |
Similar Papers
Multilingual Reasoning via Self-training (2025.naacl-long)
Copied to clipboard
| Challenge: | Recent studies have introduced eclectic strategies to improve reasoning beyond English, but these methods are related to specific language that is not always optimal for reasoning. |
| Approach: | They propose a modular approach that instructs models to structure reasoning passages in a different problem space and then self-refines their capabilities to deliver step-wise reasoning passage. |
| Outcome: | The proposed approach achieves significant improvements in multilingual reasoning of various models and task, with improved reasoning consistency across languages. |
A Survey of Multilingual Reasoning in Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | This survey provides the first in-depth review of multilingual reasoning in Language Models. |
| Approach: | This survey provides the first in-depth review of multilingual reasoning in LMs. |
| Outcome: | The present study provides the first in-depth review of multilingual reasoning in LMs. |
Disentangling Language Understanding and Reasoning Structures in Cross-lingual Chain-of-Thought Prompting (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a recent study has shown that cross-lingual chain-of-thought prompting improves learning in low-resource languages. |
| Approach: | They examine whether benefits of cross-lingual prompting arise from language-specific reasoning structures . authors employ neuron intervention and perturbation techniques to analyze and deactivate language-related reasoning neurons . |
| Outcome: | The proposed study shows that language-specific reasoning structures are essential for reasoning in each language, but have minimal effect on reasoning in other languages. |
Eliciting Better Multilingual Structured Reasoning from LLMs through Code (2024.acl-long)
Copied to clipboard
| Challenge: | xSTREET exposes a gap in base LLM performance between English and non-English reasoning tasks. |
| Approach: | They propose a multilingual structured reasoning and explanation dataset that covers four tasks across six languages and extends the English STREET benchmark to 5 additional diverse languages. |
| Outcome: | The proposed models show improved multilingual performance on scientific commonsense reasoning subtasks and no regression on non-reasoning tasks. |
Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) demonstrate impressive multilingual capability, but their performance varies substantially across different languages. |
| Approach: | They propose a generic template prompt that stimulates cross-lingual and logical reasoning skills to enhance task performance across languages. |
| Outcome: | The proposed method improves multilingual capability across languages and covers high-resource and low-resourced languages. |
Best-of-L: Cross-Lingual Reward Modeling for Mathematical Reasoning (2026.findings-eacl)
Copied to clipboard
| Challenge: | Recent studies have focused on improving reasoning ability in English models, with multilingual models receiving comparatively little attention. |
| Approach: | They propose a framework that ranks candidate reasoning traces across languages rather than within a single language. |
| Outcome: | The proposed framework improves accuracy by up to 10 points in English compared to using reward modeling within a single language. |
Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models? (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent reasoning language models (RLMs) achieve strong performance on complex reasoning tasks, yet they still exhibit a multilingual reasoning gap. |
| Approach: | They propose a strategy that incorporates an English translation into the initial reasoning trace when an understanding failure is detected. |
| Outcome: | The proposed strategy incorporates an English translation into the initial reasoning trace when an understanding failure is detected. |
Can Code-Switched Texts Activate a Knowledge Switch in LLMs? A Case Study on English-Korean Code-Switching (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large language models (LLMs) demonstrate multilingual abilities, yet they are English-centric due to dominance of English in training corpora. |
| Approach: | They propose to use a synthetic English-korean CS question-answering dataset to investigate this potential. |
| Outcome: | The proposed model can activate, identify and leverage knowledge for reasoning in low-resource languages. |
Towards a Common Understanding of Contributing Factors for Cross-Lingual Transfer in Multilingual Language Models: A Review (2023.acl-long)
Copied to clipboard
| Challenge: | Pre-trained Multilingual Language Models have shown a strong ability to transfer knowledge across languages. |
| Approach: | They examine factors contributing to the ability of MLLMs to perform zero-shot cross-lingual transfer . they identify consensuses among studies with consistent findings and resolve conflicts . |
| Outcome: | The authors outline and discuss factors that contribute to the ability of MLLMs to perform zero-shot cross-lingual transfer. |
Empowering Multi-step Reasoning across Languages via Program-Aided Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | In-context learning methods elicit Large Language Models to solve tasks using provided demonstrations without parameter updates. |
| Approach: | They propose a method for aligning reasoning programs across languages using a double-step cross-lingual prompting mechanism. |
| Outcome: | The proposed method outperforms existing prompting methods and reduces interaction time. |