Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | a dataset of imagined and recalled stories is used to study the cognitive processes involved in storytelling, contrasting imagination and recollection of events. |
| Approach: | They use a dataset of 7,000 stories to study the cognitive processes involved in storytelling, contrasting imagination and recollection of events. |
| Outcome: | The proposed measures show that imagined stories have a substantially more linear narrative flow compared to recalled stories in which adjacent sentences are more disconnected. |
Similar Papers
Are NLP Models Good at Tracing Thoughts: An Overview of Narrative Understanding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) excel in generating coherent texts, but their ability to comprehend the author’s thoughts remains uncertain. |
| Approach: | They conduct a comprehensive survey of narrative understanding tasks, examining their key features, definitions, taxonomy, associated datasets, evaluation metrics, and limitations. |
| Outcome: | The proposed framework could be extended to address novel narrative understanding tasks. |
Narrative Theory for Computational Narrative Understanding (2021.emnlp-main)
Copied to clipboard
| Challenge: | a growing body of theoretical work on narrative has been focused on the field of natural language processing . this position paper aims to provide a unifying framework for the computational study of narrative . |
| Approach: | They propose to introduce dominant theoretical frameworks to the NLP community and situate current research within distinct narratological traditions. |
| Outcome: | The proposed framework would allow for new empirical questions and applications in the field of natural language processing. |
How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMs (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models exhibit increasingly sophisticated linguistic capabilities, yet the extent to which these models reflect human-like cognition versus advanced pattern recognition remains an open question. |
| Approach: | They conduct a series of targeted experiments to assess whether LLMs construct semantic representations and pragmatic inferences in a human-like manner. |
| Outcome: | The proposed framework can be used to assess the cognitive and linguistic capabilities of large language models (LLMs). |
Fiction Flows: A Replication and Reinterpretation of Narrative Sequentiality (2026.acl-long)
Copied to clipboard
| Challenge: | a new study shows that imagined narratives exhibit higher "flow" than recalled narratives, but this advantage is not reducible to standard coherence measures. |
| Approach: | They propose a language-model-based measure of sentence-level predictability to measure narrative flow . they find that imagined stories flow better than recalled ones . |
| Outcome: | The proposed measure of sentence-level predictability is based on language models . it shows that fiction exhibits a robust sequentiality advantage over reality-bound genres . |
A Systematic Review of Reproducibility Research in Natural Language Processing (2021.eacl-main)
Copied to clipboard
| Challenge: | Despite the recent progress in reproducibility, the field is far from reaching a consensus on how reproducibility should be defined, measured and addressed. |
| Approach: | They propose to provide a wide-angle snapshot of current work on reproducibility in NLP. |
| Outcome: | The proposed work will provide a wide-angle snapshot of current work on reproducibility in NLP. |
NarraBench: A Comprehensive Framework for Narrative Benchmarking (2026.eacl-long)
Copied to clipboard
| Challenge: | Existing benchmarks for narrative understanding are poorly aligned with existing metrics. |
| Approach: | They propose to use NarraBench to assess aspects of narrative understanding that are either overlooked in current work or are poorly aligned with existing metrics. |
| Outcome: | The proposed taxonomy and survey are useful to NLP researchers . they find that only 27% of tasks are well captured by existing benchmarks . |
Text Genre and Training Data Size in Human-like Parsing (D19-1)
Copied to clipboard
| Challenge: | Using domain-specific training, NLP systems work better, but only when the training examples come from the same textual genre. |
| Approach: | They relate the states of a neural phrase-structure parser to electrophysiological measures from human participants. |
| Outcome: | The proposed model is well-matched to the training data from human participants, but only when the training examples come from the same genre. |
Mapping Brains with Language Models: A Survey (2023.findings-acl)
Copied to clipboard
| Challenge: | accumulated evidence for brain and language model activations remains ambiguous, but correlations with model size and quality provide grounds for cautious optimism. |
| Approach: | They examine the evidence accumulated by 30 studies spanning 10 datasets and 8 metrics to determine whether there is any overlap between brain and language model activations. |
| Outcome: | The findings suggest that representations extracted from NLP models can (partially) explain the signal found in neural data. |
Event-Centric Natural Language Processing (2021.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide an introduction to various methods for automating the extraction, conceptualization and prediction of events and their relations. |
| Approach: | This tutorial will provide an introduction to various methods for automating events and their relations, and a wide range of NLU and commonsense understanding tasks. |
| Outcome: | This tutorial will provide an introduction to various methods for automating extraction, conceptualization and prediction of events and their relations, and a wide range of NLU and commonsense understanding tasks. |
CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models (2024.findings-emnlp)
Copied to clipboard
Yaojia Lv, Haojie Pan, Zekun Wang, Jiafeng Liang, Yuanxing Liu, Ruiji Fu, Ming Liu, Zhongyuan Wang, Bing Qin
| Challenge: | Recent advances in large language models (LLMs) focus on replicating human cognition in specific contexts, overlooking the inherently dynamic nature of cognition. |
| Approach: | They propose a task to assess cognitive dynamics of large language models (LLMs) they introduce a benchmark and two evaluation metrics to validate the benchmark and evaluate it through participant surveys. |
| Outcome: | The proposed task overcomes the limitations of existing methods and is available for download. |