Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering (2023.acl-long)
Copied to clipboard
| Challenge: | Among recent NLP research, multi-document processing is gaining increasing attention due to the need to handle and process an increasing amount of textual data and available documents online. |
| Approach: | They propose to pre-train a generic multi-document model from a cross-document question answering pre-training objective by generating salient sentences from one document and challenging it to recover the sentence from which it was generated. |
| Outcome: | The proposed model outperforms zero-shot GPT-3.5 and GPT-4 in multiple document tasks and generates the correct answer and the salient sentence from a salient document. |
Similar Papers
Towards Multi-Document Question Answering in Scientific Literature: Pipeline, Dataset, and Evaluation (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing QA systems do not strictly enforce cross-document synthesis or exploit the explicit inter-paper structure that links sources. |
| Approach: | They propose a pipeline methodology for constructing a multi-document academic QA dataset . they detect communities based on citation networks and leverage Large Language Models . |
| Outcome: | The proposed method generates QA pairs related to multi-document content automatically and forms coherent communities based on citation networks and large language models. |
Augmenting Pre-trained Language Models with QA-Memory for Open-Domain Question Answering (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods for open-domain question-answering use an open book approach . a recent alternative is to retrieve from a collection of previously-generated question-annwer pairs . |
| Approach: | They propose a new QA system that augments a text-to-text model with a large memory of question-answer pairs and a task for the latent step of question retrieval. |
| Outcome: | The proposed system outperforms closed-book QA and can answer multi-hop questions. |
Question Answering Infused Pre-training of General-Purpose Contextualized Representations (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing pretraining objectives for question answering (QA) are not optimized for being immediately useful without fine-tuning. |
| Approach: | They propose a pre-training objective based on question answering (QA) that is based more directly on context. |
| Outcome: | The proposed model matches predictions of a more accurate cross-encoder model on 80 million synthesized QA pairs and achieves large improvements over previous state-of-the-art models on paraphrase detection and fewshot named entity recognition. |
PELMS: Pre-training for Effective Low-Shot Multi-Document Summarization (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods for abstractive multi-document summarization fail to generate concise, reflective summaries. |
| Approach: | They propose a pre-trained abstractive multi-document summarization model that uses unlabeled multi-doctoral inputs to generate concise, reflective summaries. |
| Outcome: | The proposed model outperforms competing models on a wide range of MDS datasets. |
CDLM: Cross-Document Language Modeling (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing language models (LMs) provide powerful representations for internal text structure, but there are important applications for multi-text tasks. |
| Approach: | They propose a pretraining approach that incorporates two key ideas into the masked language modeling objective. |
| Outcome: | The proposed model improves over existing models and sets of long-range transformers and can be easily applied to multiple multi-text tasks. |
Multi-Document Summarization with Centroid-Based Pretraining (2023.acl-short)
Copied to clipboard
| Challenge: | In Multi-Document Summarization, the input is a set of documents, and the output is its summary. |
| Approach: | They propose a novel pretraining objective that uses the ROUGE-based centroid of each document cluster as a proxy for its summary. |
| Outcome: | The proposed model is better or comparable to state-of-the-art models. |
Coarse-to-Fine Query Focused Multi-Document Summarization (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on query focused multi-document summarization relies heavily on retrieval-style methods. |
| Approach: | They propose a query-cluster-based model which uses more accurate modules for estimating whether text segments are relevant, likely to contain an answer, and central. |
| Outcome: | The proposed framework outperforms strong comparison systems on benchmark datasets across domains and query types. |
FewshotQA: A simple framework for few-shot learning of question answering tasks using pre-trained text-to-text models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-trained models need fine-tuning on tens of thousands of examples to achieve good results. |
| Approach: | They propose a framework that leverages pre-trained text-to-text models and aligns them with their pre-training framework. |
| Outcome: | The proposed framework outperforms the XLM-Roberta-large on multiple QA benchmarks and is applicable to multilingual situations. |
PAXQA: Generating Cross-lingual Question Answering Examples at Training Scale (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing question answering systems rely on large, high-quality training data. |
| Approach: | They propose a synthetic data generation method which decomposes cross-lingual QA into two stages . they apply a question generation model to the English side and annotation projection to translate both questions and answers. |
| Outcome: | The proposed method outperforms existing methods on cross-lingual QA datasets. |
C-MORE: Pretraining to Answer Open-Domain Questions by Consulting Millions of References (2022.acl-short)
Copied to clipboard
| Challenge: | Existing approaches to pretrain open-domain question answering systems lack task-specific annotations. |
| Approach: | They propose to pretrain a two-stage open-domain question answering system with strong transfer capabilities by using a dictionary and a large-scale corpus. |
| Outcome: | The proposed approach leads to 2%-10% gains in top-20 accuracy and improves with reader. |