| Challenge: | Existing pre-trained language models for entity-related tasks are not able to handle the nuances of procedural text. |
| Approach: | They propose to use pre-trained transformer networks to track entities in procedural text by restructuring input to focus on a particular entity. |
| Outcome: | The proposed models outperform baseline models on ingredient detection in recipes and QA over scientific processes on two different tasks. |
Similar Papers
Order-Based Pre-training Strategies for Procedural Text Understanding (2024.naacl-short)
Copied to clipboard
| Challenge: | Procedural text is difficult to understand due to the changing attributes of entities in the context. |
| Approach: | They propose sequence-based pre-training methods to enhance procedural understanding in natural language processing by using ordered instructions to guide individuals through a task. |
| Outcome: | The proposed methods improve on two datasets in the datasets NPN-Cooking and ProPara domains respectively. |
Chain and Causal Attention for Efficient Entity Tracking (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to handle entity tracking require at least log2 (n+1) layers to handle n state changes. |
| Approach: | They propose an efficient enhancement to the standard attention mechanism to handle long-term dependencies with a single layer. |
| Outcome: | The proposed model can handle entity tracking with n state changes with a single layer. |
Reasoning over Entity-Action-Location Graph for Procedural Text Understanding (2021.acl-long)
Copied to clipboard
| Challenge: | Procedural text understanding aims at tracking the states and locations of entities mentioned in a paragraph. |
| Approach: | They propose a framework to model entities-entity, action, and location relations using a graph neural network. |
| Outcome: | The proposed approach outperforms strong baselines on two datasets, ProPara and Recipes. |
Understanding Procedural Text using Interactive Entity Networks (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent efforts to track multiple entities in a procedural text treat each entity separately . e.g., scientific articles, instruction books, recipes, often contain multiple entities involved . |
| Approach: | They propose a recurrent network with memory equipped cells for state tracking . they maintain different attention matrices through specific memories to model different types of entity interactions . |
| Outcome: | The proposed model outperforms state-of-the-art models on a benchmark dataset. |
The Coreference under Transformation Labeling Dataset: Entity Tracking in Procedural Texts Using Event Models (2023.findings-acl)
Copied to clipboard
| Challenge: | et al., 2023) show that entity coreference resolution is improved when events bring about changes in entities that are not reflected in text mentions. |
| Approach: | They propose to perform transformation-based entity linking prior to coreference relation identification to improve entity coreference. |
| Outcome: | The proposed model improves coreference resolution of entities mentioned under a process-oriented model of events. |
Fine-tuning Pre-Trained Transformer Language Models to Distantly Supervised Relation Extraction (P19-1)
Copied to clipboard
| Challenge: | Current relation extraction methods suffer from noisy labels and incomplete knowledge base information. |
| Approach: | They propose a pre-trained language model that captures semantic and syntactic features and a significant amount of “common-sense” knowledge. |
| Outcome: | The proposed model achieves state-of-the-art AUC score of 0.422 on the NYT10 dataset and performs especially well at higher recall levels. |
A Dataset for Tracking Entities in Open Domain Procedural Text (2020.emnlp-main)
Copied to clipboard
Niket Tandon, Keisuke Sakaguchi, Bhavana Dalvi, Dheeraj Rajagopal, Peter Clark, Michal Guerquin, Kyle Richardson, Eduard Hovy
| Challenge: | Existing tasks require only a small set of attributes to track state changes in procedural text. |
| Approach: | They propose a task where given a procedural text as input, the task is to generate a set of state change tuples for each step. |
| Outcome: | The proposed task generates state change tuples from a set of pre-defined attributes for each step and predicts them from an open vocabulary. |
Entity Tracking in Language Models (2023.acl-long)
Copied to clipboard
| Challenge: | Existing studies on the ability of large language models to track discourse entities have not been conducted. |
| Approach: | They propose to investigate whether large language models can track entities . they first investigate whether Flan-T5, GPT-3 and GPT-3.5 can track the state of entities based on an English description of the initial state and a series of state-changing operations. |
| Outcome: | The proposed task investigates whether language models can track entities based on language descriptions and state-changing operations. |
Entity Tracking via Effective Use of Multi-Task Learning Model and Mention-guided Decoding (2023.eacl-main)
Copied to clipboard
| Challenge: | State-of-the-art entity tracking approaches either design complicated model architectures or rely on task-specific pre-training to achieve good results. |
| Approach: | They propose a multi-task learning-enabled entity tracking approach that utilizes knowledge gained from general domain tasks to improve entity tracking. |
| Outcome: | The proposed approach achieves state-of-the-art on two popular entity tracking datasets, even though it does not require any task-specific architecture design or pre-training. |
Do Syntax Trees Help Pre-trained Transformers Extract Information? (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent work suggests that incorporating syntax information from dependency trees can improve task-specific transformer models. |
| Approach: | They propose to incorporate dependency tree information into pre-trained transformers for three tasks . they propose a late fusion approach and a joint fusion technique to infuses syntax structure into attention layers. |
| Outcome: | The proposed models obtain state-of-the-art results on SRL and relation extraction tasks. |