| Challenge: | Existing studies on the ability of large language models to track discourse entities have not been conducted. |
| Approach: | They propose to investigate whether large language models can track entities . they first investigate whether Flan-T5, GPT-3 and GPT-3.5 can track the state of entities based on an English description of the initial state and a series of state-changing operations. |
| Outcome: | The proposed task investigates whether language models can track entities based on language descriptions and state-changing operations. |
Similar Papers
Effective Use of Transformer Networks for Entity Tracking (D19-1)
Copied to clipboard
| Challenge: | Existing pre-trained language models for entity-related tasks are not able to handle the nuances of procedural text. |
| Approach: | They propose to use pre-trained transformer networks to track entities in procedural text by restructuring input to focus on a particular entity. |
| Outcome: | The proposed models outperform baseline models on ingredient detection in recipes and QA over scientific processes on two different tasks. |
New or Old? Exploring How Pre-Trained Language Models Represent Discourse Entities (2022.coling-1)
Copied to clipboard
| Challenge: | Recent research shows pre-trained language models learn to encode syntactic knowledge to a certain degree. |
| Approach: | They propose to investigate the information-status of entities as discourse-new or discourse-old . they use binary classification and sequence labeling to investigate their ability to encode syntactic knowledge . |
| Outcome: | The proposed models encode information on whether an entity has been introduced before or not in the discourse. |
EvEntS ReaLM: Event Reasoning of Entity States via Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to model event implications fail to reason about the world, despite their knowledge of physical attributes. |
| Approach: | They propose to use a model prompting technique to prompt models of event implications by targeting their understanding of physical attributes. |
| Outcome: | The proposed model prompting technique is especially useful for unseen attributes or when only limited data is available. |
GPT-NER: Named Entity Recognition via Large Language Models (2025.findings-naacl)
Copied to clipboard
Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, Guoyin Wang, Chen Guo
| Challenge: | Large-scale language models (LLMs) have shown impressive ability for in-context learning with limited training data. |
| Approach: | They propose a novel sequence labeling task that transforms a sequence labeled as a text-generation task into a self-verification task that LLMs can adapt to. |
| Outcome: | The proposed model performs better on NER than supervised models on a variety of tasks . the proposed model can be easily adapted by LLMs to generate a text sequence . |
Are Large Language Model Temporally Grounded? (2024.naacl-long)
Copied to clipboard
| Challenge: | Recent large language models lack a consistent temporal model of textual narratives . sentence ordering in unlabelled texts is only weakly correlated with event ordering . |
| Approach: | They evaluate LLMs with textual narratives and evaluate their common-sense knowledge . they find that LLM models struggle the most with self-consistency . |
| Outcome: | The proposed models lack a consistent temporal model of textual narratives. |
Are LLMs Good Annotators for Discourse-level Event Relation Extraction? (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated proficiency in a wide array of natural language processing tasks, but their effectiveness over discourse-level event relation extraction tasks remains unexplored. |
| Approach: | They evaluate LLMs' ability to address discourse-level event relation extraction tasks using an open-source model and a commercial model. |
| Outcome: | The proposed model performs poorly on discourse-level event relation extraction tasks. |
A Dataset for Tracking Entities in Open Domain Procedural Text (2020.emnlp-main)
Copied to clipboard
Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal, Peter Clark, Michal Guerquin, Kyle Richardson, Eduard Hovy
| Challenge: | Existing tasks require only a small set of attributes to track state changes in procedural text. |
| Approach: | They propose a task where given a procedural text as input, the task is to generate a set of state change tuples for each step. |
| Outcome: | The proposed task generates state change tuples from a set of pre-defined attributes for each step and predicts them from an open vocabulary. |
Beyond Single-User Dialogue: Assessing Multi-User Dialogue State Tracking Capabilities of Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models have demonstrated remarkable performance in zero-shot dialogue state tracking (DST), reducing the need for task-specific training. |
| Approach: | They extend existing DST dataset by generating utterances of a second user based on speech act theory. |
| Outcome: | The proposed model incorporates utterances of a second user into conversations, enabling a controlled evaluation of LLMs in multi-user settings. |
Entity Tracking via Effective Use of Multi-Task Learning Model and Mention-guided Decoding (2023.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art entity tracking approaches either design complicated model architectures or rely on task-specific pre-training to achieve good results. |
| Approach: | They propose a multi-task learning-enabled entity tracking approach that utilizes knowledge gained from general domain tasks to improve entity tracking. |
| Outcome: | The proposed approach achieves state-of-the-art on two popular entity tracking datasets, even though it does not require any task-specific architecture design or pre-training. |
The Inner Monologue of Language Models: When Reasoning Traces Reveal More Than They Hide (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models have enabled them to tackle complex tasks . a fundamental question is: are these models aware of what they "learn" and "think"? |
| Approach: | They define three core competencies: awareness of learned latent policies, generalization of these policies across domains, alignment between internal reasoning traces and final outputs. |
| Outcome: | The results show that RL-trained models exhibit stronger generalizability to novel tasks than SFT models but weak alignment between reasoning traces and final outputs. |