Process-Level Representation of Scientific Protocols with Interactive Annotation (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing efforts to automate wet lab workflows are focusing on graph-prediction models that capture both concrete, exact quantities ("30 minutes") and vague instructions ("swirl") |
| Approach: | They manually annotate PEGs in a corpus of complex lab protocols with a novel interactive textual simulator that keeps track of entity traits and semantic constraints during annotation. |
| Outcome: | The proposed graph-prediction models are good at entity identification and local relation extraction while addressing challenges such as cross-sentence relations and long-range coreference. |
Similar Papers
An Annotated Corpus for Machine Reading of Instructions in Wet Lab Protocols (N18-2)
Copied to clipboard
| Challenge: | Existing efforts to annotate natural language instructions into machine-readable formats are limited. |
| Approach: | They propose to annotate a corpus of natural language instructions consisting of 622 wet lab protocols to facilitate automatic or semi-automatic conversion into a machine-readable format. |
| Outcome: | The proposed corpus can be used to facilitate automatic or semi-automatic conversion of protocols into a machine-readable format and benefit biological research. |
End-to-end Parsing of Procedural Text into Flow Graphs (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing flow graph parsers lack sufficient annotated data to train them . a lack of annotation can cause costly training, and poor flow graph training results in a large improvement. |
| Approach: | They propose a multi-task framework that performs tagging and graph generation simultaneously . they take advantage of the abundance of unlabelled recipes and generate noisy silver annotations . |
| Outcome: | The proposed model can unify the input representation and use compact encoders, resulting in small models with significantly fewer parameters than existing models. |
ChemReason-Bench: Benchmarking Large Language Models for Procedural Reasoning in Experimental Chemistry (2026.acl-long)
Copied to clipboard
| Challenge: | Experimental protocols in organic synthesis specify not only the intended transformation, but also an executable sequence of operations and conditions. |
| Approach: | They propose a human-validated benchmark for verifiable experimental procedure reasoning . they instantiate 7306 benchmark tasks across six complementary formats . |
| Outcome: | The proposed benchmarks show that the evaluations are less diagnostic of procedure-level decision making. |
Learning Latent Structures for Cross Action Phrase Relations in Wet Lab Protocols (2021.acl-long)
Copied to clipboard
| Challenge: | Lack of careful documentation has led to reproducibility crisis in biosciences . wet laboratory protocols are written in natural language and are unclear, composed of disconnected and distant parts and built upon implicit information that were referenced earlier or omitted entirely. |
| Approach: | They propose a model that extracts all action relationships across multiple sentences to generate a MSTG for a given protocol . they propose extending annotations in the WLP corpus for inter-sentence relations and implicit arguments to improve the model's temporal and causal relations. |
| Outcome: | The proposed model achieves an F1 score of 54.53% for temporal and causal relations in protocols from the corpus WLP-MSTG. |
API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs (2024.acl-long)
Copied to clipboard
Kinjal Basu, Ibrahim Abdelaziz, Subhajit Chaudhury, Soham Dan, Maxwell Crouse, Asim Munawar, Vernon Austel, Sadhana Kumaravel, Vinod Muthusamy, Pavan Kapanipathi, Luis Lastras
| Challenge: | Existing methods to train and test large language models that involve calls to tools and APIs are lacking. |
| Approach: | They propose a large corpora for training and systematic testing of tool-augmented LLMs. |
| Outcome: | The proposed datasets mimic real-world scenarios involving API-tasks and slot filling. |
BioGraphia: A LLM-Assisted Biological Pathway Graph Annotation Platform (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Existing methods for obtaining pathway information from biomedical literature rely on simplifying assumptions that limit their ability to capture true complexity of biological reactions. |
| Approach: | They propose a web-based platform to facilitate collaborative pathway graph annotation. |
| Outcome: | The platform supports multi-user collaboration with real-time monitoring, curation, and interactive pathway graph visualization. |
A Lightweight Modeling Middleware for Corpus Processing (L18-1)
Copied to clipboard
| Challenge: | Present-day empirical research in computational or theoretical linguistics has richly annotated and diverse corpus resources. |
| Approach: | They propose a framework for modeling arbitrary multi-modal corpus resources in a unified form for processing tools. |
| Outcome: | The proposed framework allows researchers to explore and query more diverse corpus resources and artifacts through a single interactive interface. |
Graph of Trace: Visualizing Execution Traces of Scientific Agents (2026.acl-demo)
Copied to clipboard
| Challenge: | Scientific AI agents can perform complex research tasks, but these unfolded workflows are difficult for humans to inspect and review, limiting interpretable, controllable and effective human–AI collaboration. |
| Approach: | They propose a monitoring and visualization framework that records fine-grained execution events and organizes them into a directed graph that makes agent workflows explicit as they proceed. |
| Outcome: | The proposed framework records intermediate steps (e.g. tool calls and code executions) and renders them as real-time updated visual traces that expose workflow structure. |
PAGED: A Benchmark for Procedural Graphs Extraction from Documents (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for extraction of procedural graphs from documents are not solving the task well. |
| Approach: | They propose a benchmark to test automatic extraction of procedural graphs from documents . they involve three advanced large language models and enhance them with a novel self-refine strategy . |
| Outcome: | The proposed benchmark systematically examines the progress of current methods and explores the potential of emerging large language models (LLMs) on this task. |
KCAT: A Knowledge-Constraint Typing Annotation Tool (P19-3)
Copied to clipboard
Sheng Lin, Luye Zheng, Bo Chen, Siliang Tang, Zhigang Chen, Guoping Hu, Yueting Zhuang, Fei Wu, Xiang Ren
| Challenge: | Recent years Natural Language Processing community has seen a surge of interest in fine-grained entity typing (FET) given an entity mention (i.e. a sequence of token spans representing an entity), FET aims at uncovering its contextdependent type. |
| Approach: | They propose an efficient Knowledge Constraint Fine-grained Entity Typing Annotation Tool which further improves the entity typing process through entity linking together with some practical functions. |
| Outcome: | The proposed tool improves the entity typing process by linking the candidate types with some practical functions. |