An Unsupervised Multiple-Task and Multiple-Teacher Model for Cross-lingual Named Entity Recognition (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models for named entity recognition only consider the potential transferability between two identical tasks across both domains. |
| Approach: | They propose to use a similarity metric model to improve cross-lingual named entity recognition task on target domain. |
| Outcome: | Empirical studies on 7 different languages confirm the effectiveness of the proposed model. |
Similar Papers
Single-/Multi-Source Cross-Lingual NER via Teacher-Student Learning on Unlabeled Data in Target Language (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to named entity recognition (NER) are limited by label projection with pairwise texts or direct model transfer. |
| Approach: | They propose a method where NER models in the source languages are used as teachers to train a student model on unlabeled data in the target language. |
| Outcome: | The proposed method outperforms existing state-of-the-art methods for single-source and multi-source cross-lingual NER on target languages. |
What Matters for Neural Cross-Lingual Named Entity Recognition: An Empirical Analysis (D19-1)
Copied to clipboard
| Challenge: | Named entity recognition models are challenging for languages with little training data. |
| Approach: | They propose a simple and efficient neural architecture for cross-lingual named entity recognition models. |
| Outcome: | The proposed model achieves competitive performance with the state-of-the-art on two transferable factors: sequential order and multilingual embedding. |
Multi-Grained Knowledge Distillation for Named Entity Recognition (2021.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained big models have delivered top performance in Seq2seq modeling, but their deployments in real-world applications are often hindered by excessive computations and memory demands. |
| Approach: | They propose a distillation scheme to efficiently transfer knowledge from big models to their cheaper counterparts. |
| Outcome: | The proposed scheme maximizes the assimilation of knowledge from the teacher model to the student model. |
Wider & Closer: Mixture of Short-channel Distillers for Zero-shot Cross-lingual Named Entity Recognition (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing mainstream methods for zero-shot cross-lingual named entity recognition ignore the rich and complementary information lying in the intermediate layers of pre-trained language models and domain-invariant information is easily lost during transfer. |
| Approach: | They propose a mixture of short-channel distillers to fully interact the rich hierarchical information in the teacher model and to transfer knowledge to the student model sufficiently and efficiently. |
| Outcome: | The proposed method shows great generalization and compatibility across languages and fields. |
Towards a Unified Multi-Domain Multilingual Named Entity Recognition Model (2023.eacl-main)
Copied to clipboard
Mayank Kulkarni, Daniel Preotiuc-Pietro, Karthik Radhakrishnan, Genta Indra Winata, Shijie Wu, Lingjue Xie, Shaohua Yang
| Challenge: | Named Entity Recognition is a key task whose performance is sensitive to genre and language. |
| Approach: | They propose a setup for Named Entity Recognition which includes multi-domain and multilingual training and evaluation across 13 domains and 4 languages. |
| Outcome: | The proposed model improves on 13 domains and 4 languages across 13 domain and 4 language domains. |
XtremeDistil: Multi-stage Distillation for Massive Multilingual Models (2020.acl-main)
Copied to clipboard
| Challenge: | Existing work on pre-trained language models focuses on reducing the size of these models into shallow ones. |
| Approach: | They propose a knowledge distillation technique that leverages teacher internal representations to reduce the size of pre-trained language models. |
| Outcome: | The proposed method outperforms previous methods in multilingual Named Entity Recognition (NER) it reduces the size of teacher models by 35x while retaining 95% of its F1 score. |
Cross-lingual Transfer Learning for Japanese Named Entity Recognition (N19-2)
Copied to clipboard
| Challenge: | a recent study focuses on bootstrapping named entity models from English to Japanese . TL is a technique that overcomes linguistic differences between the target and source languages . |
| Approach: | They propose to use a deep neural network model to transfer weights between languages . they also propose a novel approach that romanizes a portion of the Japanese input . |
| Outcome: | The proposed approach overcomes linguistic differences by romanizing a portion of the Japanese input. |
Cross-domain Named Entity Recognition via Graph Matching (2022.findings-acl)
Copied to clipboard
| Challenge: | Empirical results show that our method outperforms a series of transfer learning, multitask learning, and few-shot learning methods due to the data scarcity in the real-world scenario. |
| Approach: | They propose to model the label relationship as a probability distribution and construct label graphs in both source and target label spaces. |
| Outcome: | Empirical results show that the proposed method outperforms transfer learning, multi-task learning, and few-shot learning methods on four datasets. |
Cross-Domain NER using Cross-Domain Language Modeling (P19-1)
Copied to clipboard
| Challenge: | Existing methods for named entity recognition (NER) use labeled data for both source and target domains. |
| Approach: | They propose to use language modeling as a bridge between NER domains to perform cross-domain and cross-task knowledge transfer. |
| Outcome: | The proposed method extracts domain differences from cross-domain LM contrast, allowing unsupervised domain adaptation while giving state-of-the-art results among supervised domain adapters. |
Dynamic Gazetteer Integration in Multilingual Models for Cross-Lingual and Cross-Domain Named Entity Recognition (2022.naacl-main)
Copied to clipboard
| Challenge: | Named entity recognition (NER) models trained on CoNLL do not transfer well to other domains, even within the same language. |
| Approach: | They propose a token-level gating layer to augment pre-trained multilingual transformers with gazetteers containing named entities (NE) from a target language or domain. |
| Outcome: | The proposed model improves on cross-lingual transfer with an F1 score of 92.92 for English and an average of 89.43 across all languages in CoNLL. |