TDMSci: A Specialized Corpus for Scientific Literature Entity Tagging of Tasks Datasets and Metrics (2021.eacl-main)
Copied to clipboard
| Challenge: | Recent efforts to extract tasks, datasets and evaluation metrics from scientific literature focus on abstracts only. |
| Approach: | They propose a corpus that contains domain expert annotations for Task (T), Dataset (D), Metric (M) entities extracted from NLP papers. |
| Outcome: | The proposed corpus contains domain expert annotations for Task (T), Dataset (D), Metric (M) entities extracted from NLP papers. |
Similar Papers
End-to-End Construction of NLP Knowledge Graph (2021.findings-acl)
Copied to clipboard
| Challenge: | a new schema for NLP knowledge about tasks, datasets and metrics is proposed. |
| Approach: | They propose a new schema that represents knowledge about tasks, datasets and metrics in the NLP domain. |
| Outcome: | The proposed framework can be automatically built into scientific leaderboards . the proposed system achieves reasonable results for all relation types on this small-scale graph . |
Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction (P19-1)
Copied to clipboard
| Challenge: | Recent years have witnessed a significant increase in laboratory-based evaluation benchmarks in many scientific disciplines. |
| Approach: | They propose to use NLP datasets to extract task, dataset, metric and score from NLP papers to build automatic leaderboards. |
| Outcome: | The proposed model outperforms baselines in the NLP domain by a large margin. |
A Multi-level Annotated Corpus of Scientific Papers for Scientific Document Summarization and Cross-document Relation Discovery (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent studies have proposed to take advantage of the scientific paper's citation network to approach literature summarization. |
| Approach: | They propose to annotate related work sections, cite papers and sentences using machine readable data and an additional layer of papers citing the references. |
| Outcome: | The proposed corpus expands the existing data-set of related work sections and cites the papers cited in the related work section. |
SciDMT: A Large-Scale Corpus for Detecting Scientific Mentions (2024.lrec-main)
Copied to clipboard
| Challenge: | SciDMT is an enhanced and expanded corpus for scientific mention detection . existing corpora are limited by their small volume and entity linking capabilities . |
| Approach: | They propose to enhance SciDMT, an annotated scientific corpus for scientific mention detection. |
| Outcome: | The proposed corpus is the largest for scientific entity mention detection . it is based on deep learning architectures like SciBERT and GPT-3.5 . |
Beyond Metadata: What Paper Authors Say About Corpora They Use (2021.findings-acl)
Copied to clipboard
| Challenge: | Currently, dataset retrieval relies almost exclusively on metadata provided by the publishers. |
| Approach: | They propose to use metadata to extract review statements from scientific publications . they argue that a crucial piece of information is missing to inform the examination of search results . |
| Outcome: | The proposed analysis is the first of its kind in the field of Natural Language Processing. |
Tafsir Dataset: A Novel Multi-Task Benchmark for Named Entity Recognition and Topic Modeling in Classical Arabic Literature (2022.coling-1)
Copied to clipboard
Sajawel Ahmed, Rob van der Goot, Misbahur Rehman, Carl Kruse, Ömer Özsoy, Alexander Mehler, Gemma Roig
| Challenge: | Named entity recognition and topic modeling are crucial for downstream tasks in natural language processing. |
| Approach: | They propose to address named entity recognition and topic modeling on CA literature . they manually annotate the work of Tafsir Al-Tabari with span-based NEs . |
| Outcome: | The results show that the proposed task can perform state-of-the-art on historical topic models. |
The STEM-ECR Dataset: Grounding Scientific Entity References in STEM Scholarly Content to Authoritative Encyclopedic and Lexicographic Sources (2020.lrec-1)
Copied to clipboard
| Challenge: | a dataset for scientific entity extraction, classification, and resolution has been developed . a generic conceptual formalism for scientific entities is feasible, the authors say . |
| Approach: | They propose a STEM-ECR dataset that provides a domain-independent benchmark for scientific entity extraction, classification, and resolution tasks. |
| Outcome: | The proposed dataset provides a benchmark for evaluation of scientific entity extraction, classification, and resolution tasks in a domain-independent fashion. |
SciER: An Entity and Relation Extraction Dataset for Datasets, Methods, and Tasks in Scientific Documents (2024.emnlp-main)
Copied to clipboard
| Challenge: | Scientific information extraction (SciIE) is critical for converting unstructured knowledge from scholarly articles into structured data. |
| Approach: | They propose to use a scientific entity and relation extraction dataset to capture interactions between entities in full texts. |
| Outcome: | The proposed dataset captures the intricate use and interactions among entities in full texts and provides an out-of-distribution test set to offer a more realistic evaluation. |
Large Language Models for Scientific Information Extraction: An Empirical Study for Virology (2024.findings-eacl)
Copied to clipboard
| Challenge: | Scholarly communication in the digital age is facing significant challenges due to the overwhelming volume of publications. |
| Approach: | They propose to use Wikipedia infoboxes and structured Amazon product descriptions to create structured scholarly contribution summaries using text generation capabilities of LLMs. |
| Outcome: | The proposed model can be applied to complex IE tasks within terse domains like Science with 1000x fewer parameters than the state-of-the-art GPT-davinci. |
Automatic Annotation of Semantic Term Types in the Complete ACL Anthology Reference Corpus (L18-1)
Copied to clipboard
| Challenge: | a recent increase in quantitative studies of scientific text collections has led to a significant increase in the use of semantic labeling techniques. |
| Approach: | They propose to use semantic class labels to enhance a well-known resource . they use semantic labels to assign semantic class labeling to technical terms . |
| Outcome: | The proposed approach enhances the ACL Anthology Reference Corpus with semantic class labels for 20,000 technical terms . the goal is to use this information as one feature in the profiling of scientific papers, communities, and disciplines. |