Semi-supervised Domain Adaptation for Dependency Parsing via Improved Contextualized Word Representations (2020.coling-main)
Copied to clipboard
| Challenge: | Recent advances in deep neural network models have improved parsing performance on in-domain texts . however, the problem is to improve performance on out-of-domain text data when there is only a small-scale out-domain labeled data. |
| Approach: | They propose to use adversarial learning and fine-tuning BERT to improve contextualized word representations on out-of-domain texts. |
| Outcome: | The proposed models achieve consistent improvement and fine-tune BERT processes boost parsing accuracy by a large margin. |
Similar Papers
Semi-supervised Domain Adaptation for Dependency Parsing (P19-1)
Copied to clipboard
| Challenge: | Currently, most studies on cross-domain parsing focus on unsupervised domain adaptation . however, unsupervised approaches make limited progress due to the intrinsic difficulty of both domain adaptation and parse. |
| Approach: | They propose a semi-supervised domain adaptation problem for Chinese dependency parsing by using newly-annotated large-scale domain-aware datasets. |
| Outcome: | The proposed method is more effective than direct corpus concatenation and multi-task learning. |
Semi-supervised Domain Adaptation for Dependency Parsing with Dynamic Matching Network (2022.acl-long)
Copied to clipboard
| Challenge: | supervised parsing models have achieved impressive results on in-domain texts, but their performances drop drastically on out-of-domain text due to data distribution shift. |
| Approach: | They propose a dynamic matching network on the shared-private model for semi-supervised cross-domain dependency parsing. |
| Outcome: | The proposed model outperforms baseline models on all domains and achieves state-of-the-art results on all datasets. |
PERL: Pivot-based Domain Adaptation for Pre-trained Deep Contextualized Embedding Models (2020.tacl-1)
Copied to clipboard
| Challenge: | PERL is a representation learning model that uses labeled data from the source domain and unlabeled data not necessarily drawn from the target domain. |
| Approach: | They propose a model that extends contextualized word embedding models with pivot-based fine-tuning to address this bottleneck. |
| Outcome: | The proposed model outperforms strong baselines across 22 sentiment classification domain adaptation setups and improves in-domain model performance. |
Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples (P18-1)
Copied to clipboard
| Challenge: | Statistical parsers are often criticized for their performance outside of the domain they were trained on . we show that word representations reduce the need for domain adaptation when the target domain is syntactically similar to the source domain. |
| Approach: | They propose a way to adapt a parser to a syntactically similar target domain using partial annotations. |
| Outcome: | The proposed model increases the accuracy of a parser on the Wall Street Journal by 1.7% over the previous state-of-the-art model. |
Unsupervised Domain Adaptation of Contextualized Embeddings for Sequence Labeling (D19-1)
Copied to clipboard
| Challenge: | Contextualized word embeddings are becoming a ubiquitous component of natural language processing. |
| Approach: | They propose a domain-adaptive fine-tuning approach to pretrain on unlabeled text . they test this approach on sequence labeling in two challenging domains . |
| Outcome: | The proposed approach improves on sequence labeling in two domains: Early Modern English and Twitter. |
APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for dependency parsing use labeled training data for several fixed domains, but performance drops when labeles only exist for several out-domains. |
| Approach: | They propose a model for multi-source cross-domain dependency parsing that uses a parameter generation network and adversarial network for learning domain-invariant representations. |
| Outcome: | The proposed model improves cross-domain parsing performance by about 2 points over strong BERT-enhanced baselines over a recently released dataset for multi-domain dependency parse. |
Domain Adaptation with BERT-based Domain Classification and Data Selection (D19-61)
Copied to clipboard
| Challenge: | Modern deep neural models with millions of parameters can easily adapt to a new learning task and dataset when enough supervision is given. |
| Approach: | They propose a domain adaptation framework based on curriculum learning and domain-discriminative data selection. |
| Outcome: | The proposed framework outperforms discrepancy-based methods on transfer tasks while consuming only fraction of training budget. |
Representation Alignment and Adversarial Networks for Cross-lingual Dependency Parsing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have improved dependency parsing accuracy in resource-rich languages . however, the accuracy drops sharply when the model is transferred to low-resource language . |
| Approach: | They propose a representation alignment and adversarial model to filter out useful knowledge from rich-resource language and ignore useless ones. |
| Outcome: | The proposed model outperforms baseline models on the benchmark datasets by 1.37 LAS and 1.34 UAS. |
Out-of-Domain Discourse Dependency Parsing via Bootstrapping: An Empirical Analysis on Its Effectiveness and Limitation (2022.tacl-1)
Copied to clipboard
| Challenge: | Discourse parsing accuracy degrades significantly on out-of-domain text. |
| Approach: | They propose to use bootstrapping methods to adapt modern discourse dependency parsers to out-of-domain text without additional human supervision. |
| Outcome: | The proposed methods are significantly and consistently effective for unsupervised domain adaptation of discourse dependency parsing, but the low coverage of accurately predicted pseudo labels is a bottleneck for further improvement. |
Adapt in Contexts: Retrieval-Augmented Domain Adaptation via In-Context Learning (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models have demonstrated their capability with few-shot inference . however, in-domain demonstrations are not always available in real scenarios . |
| Approach: | They propose unsupervised domain adaptation problem to adapt language models from source domain to target domain without any target labels. |
| Outcome: | The proposed model performs better than baseline models on Sentiment Analysis and Named Entity Recognition tasks. |