| Challenge: | Existing statistical approaches to neural sequence labeling have been used for many tasks. |
| Approach: | They describe a toolkit for neural sequence labeling that provides a CRF inference layer for quick implementation. |
| Outcome: | The toolkit is based on PyTorch and can be run on GPUs. |
Similar Papers
Hybrid semi-Markov CRF for Neural Sequence Labeling (P18-2)
Copied to clipboard
| Challenge: | Existing conditional random fields (CRFs) use hand-crafted features to perform sequence labeling tasks. |
| Approach: | They propose to use semi-Markov conditional random fields for neural sequence labeling in natural language processing to extract features from segments instead of words. |
| Outcome: | The proposed model achieves state-of-the-art when no external knowledge is used. |
Design Challenges and Misconceptions in Neural Sequence Labeling (C18-1)
Copied to clipboard
| Challenge: | Existing neural sequence labeling models have been used for many tasks such as POS tagging, chunking and named entity recognition (NER). |
| Approach: | They propose to replicate twelve neural sequence labeling models and compare them to three benchmarks to find out which models are effective and which are inconsistent. |
| Outcome: | The proposed models are compared on NER, Chunking, and POS tagging benchmarks. |
Seq2SeqPy: A Lightweight and Customizable Toolkit for Neural Sequence-to-Sequence Modeling (2020.lrec-1)
Copied to clipboard
| Challenge: | Neural models have attracted a lot of attention in the past few years due to their complexity and need to be customized to meet specific needs. |
| Approach: | They propose a lightweight toolkit for sequence-to-sequence modeling that prioritizes simplicity and ability to customize the standard architectures easily. |
| Outcome: | The proposed tool performs similarly or even better than a very widely used sequence-to-sequence toolkit. |
CytonMT: an Efficient Neural Machine Translation Open-source Toolkit Implemented in C++ (D18-2)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) has made remarkable progress over the past few years. |
| Approach: | They propose to use C++ and NVIDIA’s GPU-accelerated libraries to build an open-source neural machine translation toolkit called CytonMT. |
| Outcome: | The proposed toolkit accelerates the training speed by 64.5% to 110.8% on neural networks of various sizes, and achieves competitive translation quality. |
NAT: Noise-Aware Training for Robust Neural Sequence Labeling (2020.acl-main)
Copied to clipboard
| Challenge: | Sequence labeling systems should perform reliably under ideal conditions and with corrupted inputs. |
| Approach: | They propose two noise-aware training objectives that improve robustness of sequence labeling performed on perturbed inputs. |
| Outcome: | The proposed methods improve robustness on English and German named entity recognition benchmarks. |
An Investigation of Potential Function Designs for Neural CRF (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to sequence labeling are based on the neural linear-chain CRF model. |
| Approach: | They propose a series of increasingly expressive potential functions for neural CRF models that integrate emission and transition functions and explicitly take contextual words as input. |
| Outcome: | The proposed model consistently achieves the best performance on the decomposed quadrilinear potential function based on the representations of two neighboring labels and two neighbored words. |
Hierarchically-Refined Label Attention Network for Sequence Labeling (D19-1)
Copied to clipboard
| Challenge: | Conditional random fields (CRF) is a powerful model for statistical sequence labeling, but it does not give much information gain over strong neural encoding. |
| Approach: | They propose a hierarchically-refined label attention network which captures potential long-term label dependency by giving each word incrementally refined label distributions with hierarchical attention. |
| Outcome: | The proposed model improves POS tagging accuracy and speeds up training and testing compared to the current model. |
NeuralClassifier: An Open-source Neural Hierarchical Multi-label Text Classification Toolkit (P19-3)
Copied to clipboard
| Challenge: | NeuralClassifier is a toolkit for hierarchical multi-label text classification. |
| Approach: | They propose a toolkit for neural hierarchical multi-label text classification . they use a variety of text encoders to implement the model . |
| Outcome: | The proposed model achieves comparable performance with reported results in the literature. |
Masked Conditional Random Fields for Sequence Labeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Conditional Random Fields (CRF) based neural models are among the most performant for sequence labeling problems, but they can sometimes generate illegal sequences of tags. |
| Approach: | They propose a conditional random field-based model that imposes restrictions on candidate paths during both training and decoding phases. |
| Outcome: | The proposed method improves on existing CRF models with near zero additional cost. |
fairseq: A Fast, Extensible Toolkit for Sequence Modeling (N19-4)
Copied to clipboard
Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, Michael Auli
| Challenge: | OpenNMT is a community-built toolkit written in multiple languages with an emphasis on extensibility. |
| Approach: | They propose to use PyTorch to train custom sequence models for translation, summarization, language modeling, and other tasks. |
| Outcome: | The proposed toolkit is fast, extensible, and useful for both research and production. |