| Challenge: | Recent advances in machine reading comprehension (MRC) have centered on literal comprehension, referring to the surface-level understanding of content. |
| Approach: | They propose a dataset specifically designed for interpretive comprehension of educational narratives, providing corresponding well-edited theme texts. |
| Outcome: | The proposed dataset spans genres and cultural origins and includes human-annotated theme keywords with varying levels of granularity. |
Similar Papers
Are NLP Models Good at Tracing Thoughts: An Overview of Narrative Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel in generating coherent texts, but their ability to comprehend the author’s thoughts remains uncertain. |
| Approach: | They conduct a comprehensive survey of narrative understanding tasks, examining their key features, definitions, taxonomy, associated datasets, evaluation metrics, and limitations. |
| Outcome: | The proposed framework could be extended to address novel narrative understanding tasks. |
MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge (L18-1)
Copied to clipboard
| Challenge: | Various approaches for script knowledge extraction and processing have been proposed in recent years. |
| Approach: | They propose a dataset to evaluate natural language understanding approaches based on commonsense knowledge. |
| Outcome: | The proposed dataset provides test cases for the broader natural language understanding community. |
Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension (2022.acl-long)
Copied to clipboard
Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
| Challenge: | Existing QA datasets rarely distinguish fine-grained reading skills, such as the understanding of varying narrative elements. |
| Approach: | They propose to use FairytaleQA to generate 10,580 questions based on 278 children-friendly stories to assess model's fine-grained learning skills. |
| Outcome: | The proposed dataset consists of 10,580 questions derived from 278 children-friendly stories, covering seven types of narrative elements or relations. |
Story Embeddings — Narrative-Focused Representations of Fictional Stories (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to model fictional narratives have focused on the aspect of "what" rather than "how" they are being told. |
| Approach: | They propose a model that embeds stories such that similar stories will result in similar embeddings. |
| Outcome: | The proposed model shows state-of-the-art performance on multiple retrieval tasks and a narrative understanding task. |
To Test Machine Comprehension, Start by Defining Comprehension (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to machine reading comprehension do not adequately define comprehension, authors argue . authors argue that existing systems are not up to the task of narrative understanding as they define it . |
| Approach: | They propose a detailed definition of comprehension for short narratives . they argue existing systems are not up to the task of narrative understanding . |
| Outcome: | The proposed task definitions suggest existing systems are not up to the task of narrative understanding as they define it. |
Scene Restoring for Narrative Machine Reading Comprehension (2020.emnlp-main)
Copied to clipboard
| Challenge: | Narrative passages describe a chain of events, which helps the machine understand the passage comprehensively. |
| Approach: | They propose a method to let machine read narrative passages with their prior knowledge . they build a scene graph using Atomic as external knowledge and encode it with GDIN . |
| Outcome: | The proposed method achieves state-of-the-art on a Story Cloze Test and CosmosQA datasets. |
Exploring Text Recombination for Automatic Narrative Level Detection (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing annotation workflows do not scale well to the annotation of complex narrative phenomena. |
| Approach: | They propose a workflow for narrative level detection that includes operationalization and a model . they propose generating training data synthetically to improve the prediction results . |
| Outcome: | The proposed workflow improves predictions by using training data synthetically. |
Narrative Theory for Computational Narrative Understanding (2021.emnlp-main)
Copied to clipboard
| Challenge: | a growing body of theoretical work on narrative has been focused on the field of natural language processing . this position paper aims to provide a unifying framework for the computational study of narrative . |
| Approach: | They propose to introduce dominant theoretical frameworks to the NLP community and situate current research within distinct narratological traditions. |
| Outcome: | The proposed framework would allow for new empirical questions and applications in the field of natural language processing. |
A Framework for Evaluation of Machine Reading Comprehension Gold Standards (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing literature on machine reading comprehension (MRC) data is limited on the data design of gold standards. |
| Approach: | They propose a framework to investigate linguistic features, lexical cues and ambiguity in MRC gold standards. |
| Outcome: | The proposed framework investigates the present linguistic features, required reasoning and background knowledge and factual correctness on the one hand, and the presence of lexical cues as a lower bound for the requirement of understanding on the other. |
text2story: A Python Toolkit to Extract and Visualize Story Components of Narrative Text (2024.lrec-main)
Copied to clipboard
| Challenge: | Story components, namely events, time, participants, and their relations, are present in narrative texts from different domains such as journalism, medicine, finance, and law. |
| Approach: | They propose to use an array of narrative extraction tools to extract narratives from text . the package contains an array and an experimental module for evaluation . |
| Outcome: | The text2story python supports the narrative extraction and visualization pipeline. |