Development of a Benchmark Corpus to Support Entity Recognition in Job Descriptions (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing tools for identifying and extracting salient entities from job descriptions are limited by the lack of publicly available training data. |
| Approach: | They propose to use a standard definition of entities and a training corpus to develop a benchmark Entity Recognition (ER) model. |
| Outcome: | The proposed model achieves an F1 score of 0.59 from 18.6k entities comprising five types (Skill, Qualification, Experience, Occupation, and Domain). |
Similar Papers
EntEval: A Holistic Evaluation Benchmark for Entity Representations (D19-1)
Copied to clipboard
| Challenge: | EntEval is a test suite of tasks that require nontrivial understanding of entities. |
| Approach: | They propose to encode the mention context or the Wikipedia hyperlink annotations to learn better entity representations. |
| Outcome: | The proposed model improves strong baselines on multiple EntEval tasks. |
ZELDA: A Comprehensive Benchmark for Supervised Entity Disambiguation (2023.eacl-main)
Copied to clipboard
| Challenge: | Entity disambiguation (ED) is the task of disambiguating named entity mentions in text to unique entries in a knowledge base. |
| Approach: | They propose a benchmark for entity disambiguation that includes a unified training data set, entity vocabulary, candidate lists and challenging evaluation splits covering 8 different domains. |
| Outcome: | The proposed benchmark is based on a unified training data set, entity vocabulary, candidate lists and evaluation splits covering 8 different domains. |
Comparing Annotated Datasets for Named Entity Recognition in English Literature (2022.lrec-1)
Copied to clipboard
| Challenge: | Generally speaking, the majority of NER tools struggle to perform well when the entities in the text contain specific characteristics. |
| Approach: | They analysed two existing annotated datasets and two additional gold standard datasets to evaluate the performance of two NER tools. |
| Outcome: | The results show that the performance of two NER tools varies significantly depending on the gold standard used for the individual evaluations. |
ELEVANT: A Fully Automatic Fine-Grained Entity Linking Evaluation and Analysis Tool (2022.emnlp-demos)
Copied to clipboard
| Challenge: | a tool for fine-grained evaluation of entity linkers on benchmarks is presented . a typical evaluation shows that overall precision and recall are poor . particular benchmarks often require very particular skills from an entity linker . |
| Approach: | They propose a tool for fine-grained evaluation of entity linkers on benchmarks . they use a graph-based tool to analyze performance of a set of entity links . |
| Outcome: | The proposed tool provides an automatic breakdown of the performance by error categories and by entity type. |
A Corpus Study and Annotation Schema for Named Entity Recognition and Relation Extraction of Business Products (L18-1)
Copied to clipboard
| Challenge: | Existing annotation guidelines for non-standard entity types and relations are lacking in news and forum texts. |
| Approach: | They propose a corpus study and an annotation schema for the annotation of product entity and company-product relation mentions. |
| Outcome: | The proposed annotation schema and guidelines are applied to the annotation of product entities and company-product relation mentions. |
A Fair and In-Depth Evaluation of Existing End-to-End Entity Linking Systems (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing evaluations of entity linking systems often lack detailed error analysis or a closer look at the results. |
| Approach: | They evaluate existing entity linking systems and propose two new benchmarks . they characterize their strengths and weaknesses and report on reproducibility aspects . |
| Outcome: | The evaluations of existing system have strong biases and artifacts . they characterize their strengths and weaknesses and report on reproducibility aspects . |
Entity Linking in the Job Market Domain (2024.findings-eacl)
Copied to clipboard
| Challenge: | In Natural Language Processing, entity linking (EL) has centered around Wikipedia, but yet remains underexplored for the job market domain. |
| Approach: | They propose to use a bi-encoder and an autoregressive model to link fine-grained span-level skill mentions to a specific taxonomy entry to quantify labor market demands. |
| Outcome: | The proposed model outperforms GENRE in strict evaluation, but performs better in loose evaluation. |
Impact of Training Instance Selection on Domain-Specific Entity Extraction using BERT (2022.naacl-srw)
Copied to clipboard
| Challenge: | Named entity recognition (NER) tasks require a large number of training examples and handcrafted features. |
| Approach: | They propose to fine-tune pre-trained language models such as BERT to achieve up to 80% F1 when fine- tuned on only 70 training examples. |
| Outcome: | The proposed model achieves 80% F1 when fine-tuned on only 70 training examples, especially on biomedical domain. |
Named Entities in Medical Case Reports: Corpus and Experiments (2020.lrec-1)
Copied to clipboard
| Challenge: | Only very few annotated corpora in the medical domain exist. |
| Approach: | They propose to annotate medical entities in case reports from PubMed Central's open access library. |
| Outcome: | The proposed corpus is the first of its kind to be made available to the scientific community in English. |
Code and Named Entity Recognition in StackOverflow (2020.acl-main)
Copied to clipboard
| Challenge: | StackOverflow has 15 million programming related questions written by 8.5 million users . however, there is a lack of fundamental NLP resources and techniques for identifying software-related named entities within natural language sentences. |
| Approach: | They propose a named entity recognition corpus for the computer programming domain with 15,372 sentences annotated with 20 fine-grained entity types. |
| Outcome: | The proposed model improves on 152 million sentences from StackOverflow and achieves 79.10 F-1 score for code and named entity recognition. |