Navigating Large-Scale Document Collections: MuDABench for Multi-Document Analytical QA

Challenge:	Existing multi-document QA benchmarks require information from only a few documents with limited cross-document reasoning.
Approach:	They propose a benchmark for multi-document analytical QA that extracts and synthesizes information across multiple documents to perform quantitative analysis.
Outcome:	The proposed approach improves both process and outcome metrics but still has bottlenecks compared to human experts.

Similar Papers

Overview

Challenge

Approach

Outcome

Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation (2025.findings-emnlp)

Copied to clipboard

Hui Huang, Julien Velcin, Yacine Kessaci

Question Answering

Challenge:	Existing QA systems do not strictly enforce cross-document synthesis or exploit the explicit inter-paper structure that links sources.
Approach:	They propose a pipeline methodology for constructing a multi-document academic QA dataset . they detect communities based on citation networks and leverage Large Language Models .
Outcome:	The proposed method generates QA pairs related to multi-document content automatically and forms coherent communities based on citation networks and large language models.

VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation (2025.naacl-long)

Copied to clipboard

Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A. Rossi, Dinesh Manocha

Multimodality

Challenge:	Existing benchmarks for document QA for visually rich documents outperform unimodal and long-context LLMs by 12-20%.
Approach:	They propose a multimodal Retrieval Augmented Generation approach that integrates visual and textual retrieval with linguistic reasoning.
Outcome:	The proposed approach outperforms unimodal and long-context LLM benchmarks for document QA by 12-20%.

M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Models (2024.findings-emnlp)

Copied to clipboard

Chuhan Li, Ziyao Shangguan, Yilun Zhao, Deyuan Li, Yixin Liu, Arman Cohan

Text Clustering

Challenge:	Existing evaluation benchmarks for foundation models in understanding scientific literature focus on single-document tasks.
Approach:	They propose a multi-modal, multi-document scientific question answering benchmark . it uses expert-annotated questions that span 70 natural language processing paper clusters .
Outcome:	The proposed benchmarks underperform human experts in multi-modal reasoning and retrieval of scientific data.

MultiDocFusion : Hierarchical and Multimodal Chunking Pipeline for Enhanced RAG on Long Industrial Documents (2025.emnlp-main)

Copied to clipboard

Joongmin Shin, Chanjun Park, Jeongbae Park, Jaehyung Seo, Heuiseok Lim

Chunking

Challenge:	Existing text chunking methods neglect complex and long industrial document structures, causing information loss and reduced answer quality.
Approach:	They propose a multimodal chunking pipeline that detects document regions and extracts text from them via OCR.
Outcome:	Extensive tests show that MultiDocFusion improves retrieval precision by 8–15% and ANLS QA scores by 2–3% compared to baselines.

WikiHowQA: A Comprehensive Benchmark for Multi-Document Non-Factoid Question Answering (2023.acl-long)

Copied to clipboard

Valeriia Bolotova-Baranova, Vladislav Blinov, Sofya Filippova, Falk Scholer, Mark Sanderson

Natural Language Interfaces

Challenge:	Answering non-factoid questions (NFQs) is a challenging task, requiring passage-level answers that are difficult to construct and evaluate.
Approach:	They propose a multi-document NFQA benchmark built on WikiHow, a website dedicated to answering “how-to” questions.
Outcome:	The proposed framework includes 11,746 human-written answers along with 74,527 supporting documents.

DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splitting (2026.acl-industry)

Copied to clipboard

Md Mofijul Islam, Md Sirajus Salekin, Nivedha Balakrishnan, Vincil C. Bishop III, Niharika Jain, Spencer Romo, Bob Strahan, Boyi Xie, Diego A. Socolinsky

Multimodality

Challenge:	Document understanding in real-world applications often requires processing heterogeneous, multi-page document packets containing multiple documents stitched together.
Approach:	They propose to use document packet splitting to improve document understanding in real-world applications.
Outcome:	The proposed datasets and evaluation metrics provide a systematic framework for advancing document understanding capabilities essential for legal, financial, healthcare, and other document-intensive domains.

Coarse-to-Fine Query Focused Multi-Document Summarization (2020.emnlp-main)

Copied to clipboard

Yumo Xu, Mirella Lapata

Text Generation

Challenge:	Existing work on query focused multi-document summarization relies heavily on retrieval-style methods.
Approach:	They propose a query-cluster-based model which uses more accurate modules for estimating whether text segments are relevant, likely to contain an answer, and central.
Outcome:	The proposed framework outperforms strong comparison systems on benchmark datasets across domains and query types.

Multi-Document Event Extraction Using Large and Small Language Models (2025.emnlp-main)

Copied to clipboard

Qingkai Min, Zitian Qu, Qipeng Guo, Xiangkun Hu, Zheng Zhang, Yue Zhang

Information Extraction & Text Mining

Challenge:	Existing approaches to multi-document event extraction have limited attention . despite its practical significance, this task has inherent challenges .
Approach:	They propose a collaborative framework that integrates large language models for multi-step reasoning and fine-tuned small language models to handle key subtasks.
Outcome:	The proposed framework outperforms existing methods and provides new insights into collaborative reasoning to tackle the complexities of multi-document event extraction.

MEBench: Benchmarking Large Language Models for Cross-Document Multi-Entity Question Answering (2025.emnlp-main)

Copied to clipboard

Teng Lin, Yuyu Luo, Honglin Zhang, Jicheng Zhang, Chunlin Liu, Kaishun Wu, Nan Tang

Question Answering

Challenge:	Large Language Models (LLMs) and Retrieval-augmented Generation (RAG) systems show promise, but their performance on cross-document MEQA remains underexplored due to the lack of tailored benchmarks.
Approach:	They propose a scalable multi-document, multi-entity benchmark to evaluate LLMs' capacity to retrieve, consolidate, and reason over scattered and dense information.
Outcome:	The proposed benchmarks show that even advanced models achieve only 59% accuracy on MEBench.

Benchmarking Deep Search over Heterogeneous Enterprise Data (2025.emnlp-industry)

Copied to clipboard

Prafulla Kumar Choubey, Xiangyu Peng, Shilpa Bhagavath, Kung-Hsiang Huang, Caiming Xiong, Chien-Sheng Wu

Information Retrieval

Challenge:	Existing methods struggle to conduct deep searches and retrieve all necessary evidence.
Approach:	They propose a benchmark for evaluating deep search, a retrieval-augmented generation that requires source-aware, multi-hop reasoning over diverse, sparsed, but related sources.
Outcome:	The proposed benchmarks show that even the best-performing agentic RAG methods achieve an average performance score of 32.96 on the benchmark.