| Challenge: | Existing metric families focus on certain aspects of sequence labeling tasks. |
| Approach: | They propose a metric that measures how much information each token contributes depending on different aspects of the sequence. |
| Outcome: | The proposed metric can satisfy all properties simultaneously. |
Similar Papers
Evaluating Extreme Hierarchical Multi-label Classification (2022.acl-long)
Copied to clipboard
| Challenge: | Several natural language processing tasks are defined as a classification problem in its most complex form: Multi-label Hierarchical Extreme classification. |
| Approach: | They propose a classification metric inspired by the Information Contrast Model (ICM) they use a set of formal properties to analyze the evaluation metrics. |
| Outcome: | The proposed evaluation metrics are suitable for multi-label hierarchical extreme classification scenarios. |
Viable Dependency Parsing as Sequence Labeling (N19-1)
Copied to clipboard
| Challenge: | Existing work on dependency parsing by sequence labeling suggested that it was impractical. |
| Approach: | They propose to use dependency trees as sequence labels to obtain fast and accurate parsers using a conventional BILSTM-based model. |
| Outcome: | The proposed models are conceptually simple, not needing traditional parsing algorithms or auxiliary structures, and provide a good speed-accuracy tradeoff, with results competitive with more complex approaches. |
A Span Selection Model for Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing models for semantic role labeling use BIO tags to predict argument spans . but performance of these approaches is weak . |
| Approach: | They propose a span-based model that takes into account all possible argument spans and scores them for each label. |
| Outcome: | The proposed model achieves state-of-the-art results on the CoNLL-2005 and 2012 datasets. |
Design Challenges and Misconceptions in Neural Sequence Labeling (C18-1)
Copied to clipboard
| Challenge: | Existing neural sequence labeling models have been used for many tasks such as POS tagging, chunking and named entity recognition (NER). |
| Approach: | They propose to replicate twelve neural sequence labeling models and compare them to three benchmarks to find out which models are effective and which are inconsistent. |
| Outcome: | The proposed models are compared on NER, Chunking, and POS tagging benchmarks. |
Embeddings of Label Components for Sequence Labeling: A Case Study of Fine-grained Named Entity Recognition (2020.acl-srw)
Copied to clipboard
| Challenge: | In general, the labels used in sequence labeling consist of different types of elements. |
| Approach: | They propose to integrate label component information as embeddings into sequence labeling models. |
| Outcome: | The proposed method improves on English and Japanese fine-grained named entity recognition on low-frequency labels. |
Bringing Emerging Architectures to Sequence Labeling in NLP (2026.eacl-long)
Copied to clipboard
| Challenge: | Pretrained Transformer encoders are the dominant approach to sequence labeling . however, few have been applied to sequence labels on flat or simplified tasks . |
| Approach: | They propose to use pretrained Transformer encoders to model relations across words . they find that the architectures adapt well across tagging tasks that vary in complexity . |
| Outcome: | The proposed architectures perform well across tagging tasks across languages and datasets. |
Dependency Graph Parsing as Sequence Labeling (2024.emnlp-main)
Copied to clipboard
| Challenge: | Various linearizations have been proposed to cast syntactic dependency parsing as sequence labeling, but they cannot handle reentrancy or cycles. |
| Approach: | They propose unbounded linearizations that can be used to cast dependency parsing as sequence labeling. |
| Outcome: | The proposed linearizations can cast syntactic dependency parsing as a sequence labeling task. |
An Empirical Study on Finding Spans (2022.emnlp-main)
Copied to clipboard
| Challenge: | Various information extraction tasks require a span finding component, which either directly yields the output or serves as an essential component of downstream linking. |
| Approach: | They propose methods for span finding, the selection of consecutive tokens in text for some downstream tasks. |
| Outcome: | The proposed methods perform better on masked language models and pre-trained encoders than on encoder-decoder models. |
SMATCH++: Standardized and Extended Evaluation of Semantic Graphs (2023.findings-eacl)
Copied to clipboard
| Challenge: | Existing graph-alignment metrics that measure graph distances are not reliable, we show . metric is spread out and does not provide upper bounds for extended tasks. |
| Approach: | They propose a metric to measure a distance between graphs by aligning nodes and counting matching graph triples. |
| Outcome: | The proposed method reduces search space and improves scoring by reducing the number of errors. |
SGM: Sequence Generation Model for Multi-label Classification (C18-1)
Copied to clipboard
| Challenge: | Existing methods ignore the correlations between labels and different parts of the text can contribute differently for predicting different labels. |
| Approach: | They propose to view the multi-label classification task as a sequence generation problem and apply a decoder-based sequence generation model to solve it. |
| Outcome: | The proposed methods outperform previous work by a substantial margin. |