Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos, Michael J.Q. Zhang, Eunsol Choi, Bhuwan Dhingra
| Challenge: | Existing methods to identify factual changes between paired documents are limited . specialized entailment-like resources and models have been applied to fact verification . |
| Approach: | They propose to represent factual changes between paired documents as question-answer pairs . they propose to generate a discriminating question given an answer span such that the question is answerable by one passage but not the other . |
| Outcome: | The proposed model can flexibly and concisely capture the updated contents of paired documents. |
Similar Papers
A Tale of Two Revisions: Summarizing Changes Across Document Versions (2024.findings-acl)
Copied to clipboard
| Challenge: | Document revision is a crucial aspect of the writing process, especially in collaborative environments where multiple authors contribute simultaneously. |
| Approach: | They propose a task of providing thematic summary of changes between document versions, organizing individual edits based on shared themes, and propose three strategies to tackle this task. |
| Outcome: | The proposed model improves its capacity to handle the task and also enables it to be used in a curated dataset. |
Generating Self-Contained and Summary-Centric Question Answer Pairs via Differentiable Reward Imitation Learning (2021.emnlp-main)
Copied to clipboard
| Challenge: | a proposed model for question-answer pairs with self-contained, summary-centric questions and length-constrained, article-summarizing answers is based on suggested question generation in conversational news recommendation systems. |
| Approach: | They propose a model for generating question-answer pairs with self-contained, summary-centric questions and length-constrained, article-summarizing answers. |
| Outcome: | The proposed model captures the central gists of the articles and achieves high answer accuracy. |
Improving Unsupervised Question Answering via Summarization-Informed Question Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Question Generation (QG) is the production of meaningful questions given a set of input passages and corresponding answers. |
| Approach: | They propose a method which uses questions generated heuristically from news summaries as a source of training data for a QG system. |
| Outcome: | The proposed method outperforms previous unsupervised models on three in-domain datasets and three out-of-domain ones. |
DiFRa: A Unified Framework for Harmonizing Semantic Diversity and Factual Consistency in Question-Answer Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Question-Answer Generation (QAG) is essential for domain-specific large language models post-training. |
| Approach: | They propose a framework that balances semantic diversity and factual consistency . they propose entropy and consistency scores that harmonize the trade-off between diversity and correctness . |
| Outcome: | The proposed framework outperforms baseline models in generating diverse QA pairs . the proposed framework harmonizes semantic entropy and consistency scores to quantify trade-off between diversity and correctness. |
Consecutive Question Generation via Dynamic Multitask Learning (2022.findings-emnlp)
Copied to clipboard
| Challenge: | , . ; ) ()((); ()) .())((2): ""(). |
| Approach: | They propose a sequential sequential question-answer generation task with auxiliary tasks generating other elements to generate logically related question-anchor pairs to understand a passage. |
| Outcome: | The proposed framework improves question generation significantly and benefit multiple related tasks. |
Asking and Answering Questions to Evaluate the Factual Consistency of Summaries (2020.acl-main)
Copied to clipboard
| Challenge: | Existing automatic evaluation metrics for summarization are insensitive to factual inconsistencies. |
| Approach: | They propose an automatic evaluation protocol that detects factual inconsistencies in a model-generated summary. |
| Outcome: | QAGS has higher correlations with human judgments of factual consistency than other evaluation metrics. |
A guide to the dataset explosion in QA, NLI, and commonsense reasoning (2020.coling-tutorials)
Copied to clipboard
| Challenge: | a tutorial aims to provide an up-to-date guide to the recent datasets . the target audience is the NLP practitioners who are lost in dozens of the recent data sets. |
| Approach: | This tutorial provides an up-to-date guide to the recent datasets . it surveys old and new methodological issues with dataset construction . |
| Outcome: | This tutorial aims to provide an up-to-date guide to the recent datasets . it surveys the old and new methodological issues with dataset construction . |
QAEVENT: Event Extraction as Question-Answer Pairs Generation (2024.findings-eacl)
Copied to clipboard
| Challenge: | a novel approach to document-level event extraction requires linguistic expertise. |
| Approach: | They propose a question-and-answer-pair representation of document-level events as question and answer pairs (QAEVENT) this allows for more scalable and faster annotations from crowdworkers without linguistic expertise. |
| Outcome: | The proposed method produces high-quality data for document-level event extraction compared to pre-defined schemas . it is scalable and faster than existing methods, with no linguistic expertise. |
Adaptive Question Answering: Enhancing Language Model Proficiency for Addressing Knowledge Conflicts with Source Citations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on citation generation has focused on unambiguous settings with single answers, failing to address the complexity of real-world scenarios. |
| Approach: | They propose a task of QA with source citation in ambiguous settings where multiple valid answers exist, where multiple sources exist. |
| Outcome: | The proposed framework generates multiple answers and cites their sources, allowing users to verify the factuality of each answer and make informed decisions. |
Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors (2023.acl-long)
Copied to clipboard
Liyan Tang, Tanya Goyal, Alex Fabbri, Philippe Laban, Jiacheng Xu, Semih Yavuz, Wojciech Kryscinski, Justin Rousseau, Greg Durrett
| Challenge: | Abstractive summarization systems still include factual errors in generated summaries despite recent improvements in factuality detection . |
| Approach: | They aggregate factuality error annotations from nine existing datasets and stratify them according to the underlying summarization model. |
| Outcome: | The proposed method improves on the ChatGPT-based model and shows that it is not superior for all error types. |