Training for Gibbs Sampling on Conditional Random Fields with Neural Scoring Factors (2020.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in NLP focus on simple approaches to model the output label space . graphical models are often limited to (heuristic) greedy search and its variants . |
| Approach: | They propose an approach for efficiently training and decoding hybrids of graphical and graphical models based on Gibbs sampling. |
| Outcome: | The proposed approach improves on Dutch and Dutch with graphical models . the proposed model improves over a strong baseline on three languages . |
Similar Papers
Constrained Decoding for Computationally Efficient Named Entity Recognition Taggers (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Named entity recognition models use a conditional random field as the final layer . current work eschews prior knowledge of how the span encoding scheme works . |
| Approach: | They propose to constrain the output to suppress illegal transitions to train a tagger with a cross-entropy loss twice as fast as a CRF. |
| Outcome: | The proposed model trains twice as fast as a CRF with statistically insignificant differences in F1 . the proposed model is open source and can be used in PyTorch and TensorFlow. |
Hybrid semi-Markov CRF for Neural Sequence Labeling (P18-2)
Copied to clipboard
| Challenge: | Existing conditional random fields (CRFs) use hand-crafted features to perform sequence labeling tasks. |
| Approach: | They propose to use semi-Markov conditional random fields for neural sequence labeling in natural language processing to extract features from segments instead of words. |
| Outcome: | The proposed model achieves state-of-the-art when no external knowledge is used. |
Masked Conditional Random Fields for Sequence Labeling (2021.naacl-main)
Copied to clipboard
| Challenge: | Conditional Random Fields (CRF) based neural models are among the most performant for sequence labeling problems, but they can sometimes generate illegal sequences of tags. |
| Approach: | They propose a conditional random field-based model that imposes restrictions on candidate paths during both training and decoding phases. |
| Outcome: | The proposed method improves on existing CRF models with near zero additional cost. |
Neural Language Modeling for Named Entity Recognition (2020.coling-main)
Copied to clipboard
| Challenge: | Experimental results show that named entity recognition systems are faster and more flexible for the size of the corpus. |
| Approach: | They propose to use a neural language model as an alternative to the conditional random field layer for named entity recognition. |
| Outcome: | The proposed system has a significant speed advantage with a marginal performance degradation. |
Entity Disambiguation on a Tight Labeling Budget (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to training entity disambiguation models require a small labeling budget . a defense research analyst might need to map military equipment to a knowledge base describing emergent defense technologies. |
| Approach: | They propose a method that combines feature diversity with low rank correction . they use bilinear tensor models to train a model that uses a rich representation of context . |
| Outcome: | The proposed approach reduces the amount of labeled data necessary to achieve a given performance. |
Bridging Pre-trained Language Models and Hand-crafted Features for Unsupervised POS Tagging (2022.findings-acl)
Copied to clipboard
| Challenge: | Large-scale pre-trained language models (PLMs) have made extraordinary progress in most NLP tasks, but they fail to achieve state-of-the-art (SOTA) performance. |
| Approach: | They propose a Guassian HMM variant for unsupervised POS tagging that incorporates contexualized word representations into the decoder. |
| Outcome: | The proposed model outperforms state-of-the-art models on Penn Treebank and multilingual Universal Dependencies treebank v2.0. |
Filtered Semi-Markov CRF (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for sequence labeling tasks such as Named Entity Recognition (NER) suffer from quadratic complexity over sequence length and poor performance compared to CRF. |
| Approach: | They propose a variant of Semi-Markov CRF that incorporates a filtering step to eliminate irrelevant segments, reducing complexity and search space. |
| Outcome: | The proposed method outperforms both CRF and Semi-CRF on several NER benchmarks while being significantly faster. |
Learning Named Entity Tagger using Domain-Specific Dictionary (D18-1)
Copied to clipboard
| Challenge: | Existing methods to build reliable named entity recognition systems require large amounts of manually-labeled training data. |
| Approach: | They propose a revised fuzzy CRF layer to handle tokens with multiple possible labels to address noisy distant supervision. |
| Outcome: | The proposed model can handle tokens with multiple possible labels under the traditional framework and improves on the existing model with a new Tie or Break scheme. |
BERTifying the Hidden Markov Model for Multi-Source Weakly Supervised Named Entity Recognition (2021.acl-long)
Copied to clipboard
| Challenge: | Existing NER models are supervised by a large number of training sequences, each pre-annotated with token-level labels. |
| Approach: | They propose a conditional hidden Markov model which can effectively infer true labels from multi-source noisy labels in an unsupervised way. |
| Outcome: | The proposed model outperforms state-of-the-art weakly supervised NER models on four benchmarks from various domains. |
Towards Improving Neural Named Entity Recognition with Gazetteers (P19-1)
Copied to clipboard
| Challenge: | Currently, neural models for named entity recognition are based on data-driven models, with a strong emphasis on getting rid of the efforts for collecting external resources or designing hand-crafted features. |
| Approach: | They propose to use external gazetteers to efficiently access annotated data to generalize beyond the annotation of entities. |
| Outcome: | The proposed model can access external gazetteers while avoiding the effort to design hand-crafted features. |