LAReQA: Language-Agnostic Answer Retrieval from a Multilingual Pool

Challenge:	LAReQA tests for “strong” cross-lingual alignment, requiring semantically related cross-language pairs to be closer in representation space than unrelated same-language pair.
Approach:	They propose a new benchmark for language-agnostic answer retrieval from a multilingual candidate pool that tests for "strong" cross-lingual alignment . they augment training data via machine translation and find that model performance is improved by augmenting training data through machine translation .
Outcome:	The proposed task is based on multilingual BERT (mBERT) and XLM-R.

Similar Papers

Overview

Challenge

Approach

Outcome

XLQA: A Benchmark for Locale-Aware Multilingual Open-Domain Question Answering (2025.emnlp-main)

Copied to clipboard

Keonwoo Roh, Yeong-Joon Ju, Seong-Whan Lee

Natural Language Interfaces

Challenge:	Large Language Models (LLMs) have shown significant progress in Open-domain question answering (ODQA) but most evaluations focus on English and assume locale-invariant answers across languages.
Approach:	They propose a benchmark specifically designed for locale-sensitive multilingual ODQA that uses 3,000 English seed questions expanded to eight languages.
Outcome:	The proposed benchmarks are based on 3,000 English seed questions expanded to eight languages and a human-verified annotation distinguishing locale-invariant and locale-sensitive cases.

M3TQA: Massively Multilingual Multitask Table Question Answering (2026.findings-acl)

Copied to clipboard

Daixin Shu, Jian Yang, Zhenhe Wu, Xianjie Wu, Xianfu Cheng, Guan Xiangyuan, Yanghai Wang, Pengfei Wu, Tingyang Yang, Hualei Zhu, Wei Zhang, Ge Zhang, Jiaheng Liu, Zhoujun Li

Structured Data in NLP

Challenge:	Existing multilingual table benchmarks suffer from geolinguistic imbalance - overrepresenting certain languages and lacking sufficient scale for rigorous cross-lingual analysis.
Approach:	They propose a framework for massively multilingual table question answering that includes tables expanded to 97 languages from Chinese and English sources.
Outcome:	Experiments on state-of-the-art LLMs show that synthetically generated training data significantly boosts performance, especially for low-resource languages.

MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2021.tacl-1)

Copied to clipboard

Shayne Longpre, Yi Lu, Joachim Daiber

Natural Language Interfaces

Challenge:	Existing multilingual QA datasets lack linguistic diversity and comparable evaluation between languages.
Approach:	They propose a multilingual question-answer evaluation set with 10k English queries and human translations of them into 25 additional languages and dialects.
Outcome:	The proposed model is based on a multilingual knowledge questions and answers evaluation set with 26 languages.

XOR QA: Cross-lingual Open-Retrieval Question Answering (2021.naacl-main)

Copied to clipboard

Akari Asai, Jungo Kasai, Jonathan Clark, Kenton Lee, Eunsol Choi, Hannaneh Hajishirzi

Question Answering

Challenge:	a dataset of 40k information-seeking questions across seven languages is used to answer multilingual question answering tasks.
Approach:	They propose a task framework that allows questions from one language to be answered via answer content from another language.
Outcome:	The proposed framework can be used to answer questions from one language to another . the dataset was built on 40K questions across 7 languages, but could not find same-language answers .

Boosting Data Utilization for Multilingual Dense Retrieval (2025.emnlp-main)

Copied to clipboard

Chao Huang, Fengran Mo, Yufeng Chen, Changhao Guan, Zhenrui Yue, Xinyu Wang, Jinan Xu, Kaiyu Huang

Information Retrieval

Challenge:	Existing studies focus on fine-tuning multilingual dense retrieval models, but data scarcity for low-resource languages makes it difficult to align representations in a shared vector space.
Approach:	They propose to obtain high-quality hard negative samples and effective mini-batch data to boost data utilization for multilingual dense retrieval by obtaining high-quality negative samples.
Outcome:	The proposed method outperforms existing baselines on a multilingual retrieval benchmark, MIRACL, with 16 languages.

XQA: A Cross-lingual Open-domain Question Answering Dataset (P19-1)

Copied to clipboard

Jiahua Liu, Yankai Lin, Zhiyuan Liu, Maosong Sun

Natural Language Interfaces

Challenge:	Open-domain question answering aims to answer questions through text retrieval and reading comprehension . but, the success of these models relies on a massive volume of training data, which is not available in other languages . a new dataset aims at investigating cross-lingual OpenQA .
Approach:	They propose to use a dataset for cross-lingual OpenQA research to test models . they use XQA dataset to train models with large volumes of labeled data .
Outcome:	The proposed model achieves best results in almost all target languages while the performance is lower than that of English.

MINERS: Multilingual Language Models as Semantic Retrievers (2024.findings-emnlp)

Copied to clipboard

Genta Winata, Ruochen Zhang, David Adelani

Semantic Text Processing

Challenge:	Existing benchmarks have evaluated language models to evaluate their performance across a range of embedding tasks.
Approach:	They propose a benchmark to evaluate the robustness of multilingual language models in semantic retrieval tasks including bitext mining and classification via retrieval-augmented contexts.
Outcome:	The proposed framework evaluates the robustness of multilingual LMs in retrieval tasks across over 200 languages, including extremely low-resource languages in challenging cross-lingual and code-switching settings.

MULTITAT: Benchmarking Multilingual Table-and-Text Question Answering (2025.findings-emnlp)

Copied to clipboard

Xuanliang Zhang, Dingzirui Wang, Keyan Xu, Qingfu Zhu, Wanxiang Che

Multimodality

Challenge:	Existing TATQA datasets are limited to English, leading to drawbacks . existing datasets overlook challenges of multilingual TAT-QA and do not reflect real-world multilingual scenarios .
Approach:	They propose a multilingual TATQA dataset that can be translated into 10 languages . they use data from 3 mainstream TATQ datasets and analyze the results .
Outcome:	The proposed dataset outperforms other baselines by an average of 3.3 .

TableEval: A Real-World Benchmark for Complex, Multilingual, and Multi-Structured Table Question Answering (2025.emnlp-main)

Copied to clipboard

Junnan Zhu, Jingyi Wang, Bohan Yu, Xiaoyu Wu, Junbo Li, Lei Wang, Nan Xu

Structured Data in NLP

Challenge:	Existing TableQA benchmarks focus on simple flat tables and suffer from data leakage . current benchmarks are monolingual and fail to capture cross-lingual variability .
Approach:	They propose a table-based TableQA benchmark to evaluate LLMs on real-world tasks.
Outcome:	The proposed benchmarks show that they achieve high agreement with human judgment . the proposed framework improves on the alignment between model responses and reference answers .

Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)

Copied to clipboard

Katharina Hämmerl, Jindřich Libovický, Alexander Fraser

Language Models

Challenge:	Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models.
Approach:	They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key .
Outcome:	The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key .