Unlocking Markets: A Multilingual Benchmark to Cross-Market Question Answering (2024.emnlp-main)
Copied to clipboard
| Challenge: | Product-related question answering (PQA) involves utilizing product-related resources to provide precise answers to users. |
| Approach: | They propose a task of multilingual cross-market product-based question answering that combines product-related questions with product-specific questions from a multilingual marketplace. |
| Outcome: | The proposed task provides answers to product-related questions in a multilingual marketplace even in fewer languages. |
Similar Papers
xPQA: Cross-Lingual Product Question Answering in 12 Languages (2023.acl-industry)
Copied to clipboard
| Challenge: | Existing work on product question answering systems focuses mainly on English, but in practice there is need to support multiple customer languages while leveraging product information available in English. |
| Approach: | They present a large-scale annotated cross-lingual PQA dataset in 12 languages and evaluate three approaches to generating a natural-sounding non-English answer. |
| Outcome: | The proposed dataset supports crosslingual product question answering (PQA) systems that provide answers to customers’ questions as they shop for products. |
MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing multilingual QA datasets lack linguistic diversity and comparable evaluation between languages. |
| Approach: | They propose a multilingual question-answer evaluation set with 10k English queries and human translations of them into 25 additional languages and dialects. |
| Outcome: | The proposed model is based on a multilingual knowledge questions and answers evaluation set with 26 languages. |
XLQA: A Benchmark for Locale-Aware Multilingual Open-Domain Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown significant progress in Open-domain question answering (ODQA) but most evaluations focus on English and assume locale-invariant answers across languages. |
| Approach: | They propose a benchmark specifically designed for locale-sensitive multilingual ODQA that uses 3,000 English seed questions expanded to eight languages. |
| Outcome: | The proposed benchmarks are based on 3,000 English seed questions expanded to eight languages and a human-verified annotation distinguishing locale-invariant and locale-sensitive cases. |
Answering Product-related Questions with Heterogeneous Information (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing product question answering methods only consider a single information source such as user reviews and/or require large amounts of labeled data. |
| Approach: | They propose a framework to exploit heterogeneous information including natural language text and attribute-value pairs from two information sources of the concerned product, namely product details and user reviews. |
| Outcome: | The proposed framework achieves superior performance over state-of-the-art models on a real-world dataset. |
MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) and Retrieval-augmented Generation (RAG) systems show promise, but their performance on cross-document MEQA remains underexplored due to the lack of tailored benchmarks. |
| Approach: | They propose a scalable multi-document, multi-entity benchmark to evaluate LLMs' capacity to retrieve, consolidate, and reason over scattered and dense information. |
| Outcome: | The proposed benchmarks show that even advanced models achieve only 59% accuracy on MEBench. |
M2QA: Multi-domain Multilingual Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Language varies along several axes, most importantly, language instance and domain . lack of evaluation datasets prevents transfer of NLP systems to non-dominant languages . |
| Approach: | They propose a multi-domain multilingual question answering benchmark to explore cross-lingual cross-domain performance of fine-tuned models and state-of-the-art LLMs. |
| Outcome: | The proposed benchmark compared 13,500 SQuAD 2.0-style question-answer instances in German, Turkish, and Chinese for the domains of product reviews, news, and creative writing. |
xGQA: Cross-Lingual Visual Question Answering (2022.findings-acl)
Copied to clipboard
Jonas Pfeiffer, Gregor Geigle, Aishwarya Kamath, Jan-Martin Steitz, Stefan Roth, Ivan Vulić, Iryna Gurevych
| Challenge: | a lack of multilingual multimodal datasets has hindered multimodal vision and language modeling efforts. |
| Approach: | They propose a multilingual evaluation benchmark for the visual question answering task . they extend the established English GQA dataset to 7 typologically diverse languages . |
| Outcome: | The proposed methods outperform current state-of-the-art models in zero-shot cross-lingual settings, but the accuracy remains low across languages. |
MLQA: Evaluating Cross-lingual Extractive Question Answering (2020.acl-main)
Copied to clipboard
| Challenge: | Question answering (QA) models have shown rapid progress enabled by the availability of large, high-quality benchmark datasets. |
| Approach: | They present a multi-way aligned extractive QA evaluation benchmark in 7 languages . they evaluate state-of-the-art cross-lingual models and machine-translation-based baselines . |
| Outcome: | The proposed model is based on MLQA, which has over 12K instances in english and 5K in each other language. |
Product Question Answering in E-Commerce: A Survey (2023.acl-long)
Copied to clipboard
| Challenge: | Product question answering (PQA) aims to automatically provide instant responses to customer’s questions in E-commerce platforms. |
| Approach: | They categorize PQA studies into four problem settings in terms of the form of provided answers. |
| Outcome: | The proposed methods capture the unique challenges of product question answering (PQA) . |
XOR QA: Cross-lingual Open-Retrieval Question Answering (2021.naacl-main)
Copied to clipboard
| Challenge: | a dataset of 40k information-seeking questions across seven languages is used to answer multilingual question answering tasks. |
| Approach: | They propose a task framework that allows questions from one language to be answered via answer content from another language. |
| Outcome: | The proposed framework can be used to answer questions from one language to another . the dataset was built on 40K questions across 7 languages, but could not find same-language answers . |