SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Scientific information extraction (SciIE) is critical for converting unstructured knowledge from scholarly articles into structured data. |
| Approach: | They propose to use a scientific entity and relation extraction dataset to capture interactions between entities in full texts. |
| Outcome: | The proposed dataset captures the intricate use and interactions among entities in full texts and provides an out-of-distribution test set to offer a more realistic evaluation. |
Similar Papers
POLYIE: A Dataset of Information Extraction from Polymer Material Scientific Literature (2024.naacl-long)
Copied to clipboard
Jerry Cheung, Yuchen Zhuang, Yinghao Li, Pranav Shetty, Wantian Zhao, Sanjeev Grampurohit, Rampi Ramprasad, Chao Zhang
| Challenge: | SciIE datasets for polymer materials are lacking for this class of materials . POLYIE is curated from 146 full-length polymer scholarly articles . |
| Approach: | They propose a SciIE dataset for polymer materials that uses entity annotations from 146 full-length articles. |
| Outcome: | The proposed dataset is curated from 146 full-length polymer scholarly articles . it presents challenges due to diverse lexical formats of entities and ambiguity between entities . |
CitationIE: Leveraging the Citation Graph for Scientific Information Extraction (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on scientific information extraction (SciIE) considers extraction solely based on the content of an individual paper, without considering the paper’s place in the broader literature. |
| Approach: | They propose to automate the extraction of key information from scientific documents by leveraging a complementary source: the citation graph of referential links between citing and cited papers. |
| Outcome: | The proposed model improves on a set of English-language scientific documents. |
SciNLP: A Domain-Specific Benchmark for Full-Text Scientific Entity and Relation Extraction in NLP (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets for structured information extraction focus on specific publication sections due to domain complexity and high cost of annotating scientific texts. |
| Approach: | They propose a specialized benchmark for full-text entity and relation extraction in the natural language processing domain. |
| Outcome: | The proposed dataset comprises 60 manually annotated full-text NLP publications covering 7,072 entities and 1,826 relations. |
Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction (D18-1)
Copied to clipboard
| Challenge: | Existing relation extraction systems are designed for within-sentence relations, but extracting information from scientific articles requires relations across sentences. |
| Approach: | They propose a multi-task setup for identifying entities, relations, and coreference clusters in scientific articles . they develop a unified framework called SciIE with shared span representations to solve this problem . |
| Outcome: | The proposed model outperforms existing models without domain-specific features in scientific information extraction. |
SciEvent: Benchmarking Multi-domain Scientific Event Extraction (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on scientific information extraction relies on entity-relation extraction in narrow domains . current models struggle in domains such as sociology and humanities . |
| Approach: | They propose a multi-domain benchmark for scientific abstract annotations using a unified event extraction schema. |
| Outcome: | The proposed benchmark includes 500 abstracts across five research domains with manual annotations of event segments, triggers, and fine-grained arguments. |
What Do You Mean by Relation Extraction? A Survey on Datasets and Study on Scientific Relation Classification (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing RE surveys focus on modeling techniques, but there are few that are based on real-world scenarios. |
| Approach: | They propose to survey RE datasets and revisit the task definition and its adoption by the community. |
| Outcome: | The proposed approach improves the reliability of RE evaluations across multiple datasets and reveals significant discrepancies in annotations. |
ITER: Iterative Transformer-based Entity Recognition and Relation Extraction (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in NLP generate structured information in an autoregressive manner, causing low throughput . authors propose an efficient encoder-based relation extraction model that performs the task in three parallelizable steps. |
| Approach: | They propose an efficient encoder-based relation extraction model that performs the task in three parallelizable steps. |
| Outcome: | The proposed model achieves state-of-the-art on two datasets and is faster than existing models. |
SciREX: A Challenge Dataset for Document-Level Information Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Conventional datasets and methods for information extraction focus on within-sentence relations from general Newswire text. |
| Approach: | They propose a document-level IE dataset that integrates automatic and human annotations to annotate entities and document- level N-ary relation identification from scientific articles. |
| Outcome: | The proposed dataset extends state-of-the-art IE models to document-level IE. |
The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources (2020.lrec-1)
Copied to clipboard
| Challenge: | a dataset for scientific entity extraction, classification, and resolution has been developed . a generic conceptual formalism for scientific entities is feasible, the authors say . |
| Approach: | They propose a STEM-ECR dataset that provides a domain-independent benchmark for scientific entity extraction, classification, and resolution tasks. |
| Outcome: | The proposed dataset provides a benchmark for evaluation of scientific entity extraction, classification, and resolution tasks in a domain-independent fashion. |
GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasets (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Named Entity Recognition (NER) models are crucial for academic writing . existing ground truth datasets do not treat fine-grained types like ML model and model architecture as separate entity types . |
| Approach: | They propose to annotate 100 full-text scientific publications and a first baseline model for 10 entity types centered around ML models and datasets. |
| Outcome: | The proposed model can be used to identify 10 entity types in scientific articles . existing models cannot recognize fine-grained models like ML models and model architecture . |