| Challenge: | Pre-trained word embeddings and self-training have been used in dependency parsing tasks for years. |
| Approach: | They compare tri-training and pretrained word embeddings in dependency parsing . they use language-specific FastText and ELMo embedds and multilingual BERT embedders . |
| Outcome: | The proposed methods are tri-training and pretrained word embeddings. |
Similar Papers
Combining Deep Generative Models and Multi-lingual Pretraining for Semi-supervised Document Classification (2021.eacl-main)
Copied to clipboard
| Challenge: | Semi-supervised learning and multilingual pretraining have been shown to be effective for task-specific labelled data shortages. |
| Approach: | They propose to combine semi-supervised deep generative models and multi-lingual pretraining to form a pipeline for document classification task. |
| Outcome: | The proposed method outperforms state-of-the-art models in low-resource settings across several languages and outperformed existing models in English. |
Treebank Embedding Vectors for Out-of-Domain Dependency Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | a recent advance in monolingual dependency parsing is the idea of a treebank embedding vector . this allows the model to prefer training data from one treebank over another at test time . |
| Approach: | They propose a method to predict a treebank vector for sentences that do not come from a particular treebank . they also explore what happens when they move away from predefined treebank embedding vectors . |
| Outcome: | The proposed method can predict treebank vectors for sentences that do not come from a treebank used in training with sufficient accuracy for nine out of ten languages. |
Parser Training with Heterogeneous Treebanks (P18-2)
Copied to clipboard
| Challenge: | In the 2017 CoNLL Shared Task on Universal Dependency Parsing, 25 languages have more than one treebank . many teams did not take advantage of the multiple treebanks, however, and trained one model per treebank instead of one model for each language. |
| Approach: | They propose a method to make the most of heterogeneous treebanks when training a monolingual parser. |
| Outcome: | The proposed method improves on training with multiple treebanks for a single language. |
A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages (2020.acl-main)
Copied to clipboard
| Challenge: | a recent trend in neural NLP has been the introduction of feature-based and fine-tuning methods . we train monolingual contextualized word embeddings for five mid-resource languages . |
| Approach: | They use common Crawl corpus to train monolingual contextualized word embeddings . they compare performance of OSCAR-based and Wikipedia-based embeddables on part-of-speech tasks . |
| Outcome: | The results show that OSCAR-based and Wikipedia-based embeddings perform better than Wikipedia-style embedders on part-of-speech tagging and parsing tasks. |
Semi-supervised Domain Adaptation for Dependency Parsing (P19-1)
Copied to clipboard
| Challenge: | Currently, most studies on cross-domain parsing focus on unsupervised domain adaptation . however, unsupervised approaches make limited progress due to the intrinsic difficulty of both domain adaptation and parse. |
| Approach: | They propose a semi-supervised domain adaptation problem for Chinese dependency parsing by using newly-annotated large-scale domain-aware datasets. |
| Outcome: | The proposed method is more effective than direct corpus concatenation and multi-task learning. |
Multilingual Dependency Parsing for Low-Resource Languages: Case Studies on North Saami and Komi-Zyrian (L18-1)
Copied to clipboard
| Challenge: | Developing systems for low-resource languages is a crucial issue for Natural Language Processing (NLP). |
| Approach: | They propose a method for parsing low-resource languages with very small training corpora using multilingual word embeddings and annotated corporata of larger languages. |
| Outcome: | The proposed method improves dependency parsing for low-resource languages with very small training corpora compared to previous work . it also explores whether contemporary contact languages or genetically related languages would be the most fruitful starting point for multilingual parsers. |
Quantifying training challenges of dependency parsers (C18-1)
Copied to clipboard
| Challenge: | a new metric is introduced to evaluate the difficulty to learn a given class of dependencies . a series of systematic computations using that metric have revealed interesting properties of the 3 considered parsing algorithms . |
| Approach: | They introduce a new metric to evaluate the difficulty to learn a given class of dependencies . they use it to characterize the information conveyed by cross-lingual parsers . |
| Outcome: | The proposed metric reveals the kind of dependencies that require high effort during training . it also shows that cross-lingual parsers can provide better quality information . |
A systematic comparison of methods for low-resource dependency parsing on genuinely low-resource languages (D19-1)
Copied to clipboard
| Challenge: | Large annotated treebanks are available for only a tiny fraction of the world's languages, and there is a wealth of literature on strategies for parsing with few resources. |
| Approach: | They propose three strategies for improving low-resource parsers: data augmentation, cross-lingual training, and transliteration. |
| Outcome: | The proposed methods improve low-resource parsers by using data augmentation, cross-lingual training, and transliteration. |
Go Simple and Pre-Train on Domain-Specific Corpora: On the Role of Training Data for Text Classification (2020.coling-main)
Copied to clipboard
| Challenge: | Pre-trained language models provide the foundations for state-of-the-art performance across a wide range of natural language processing tasks, including text classification. |
| Approach: | They compare the performance of a linear classifier based on word embeddings with a pre-trained language model, i.e., BERT, across a wide range of datasets and classification tasks. |
| Outcome: | The proposed method outperforms baselines in standard datasets with large training sets, but in settings with small training datasets it performs better. |
Cross-lingual Parsing with Polyglot Training and Multi-treebank Learning: A Faroese Case Study (D19-61)
Copied to clipboard
| Challenge: | Cross-lingual dependency parsing involves transferring syntactic knowledge from one language to another. |
| Approach: | They compare two approaches to cross-lingual dependency parsing using monolingual source models and a polyglot model which is trained on the combination of all source languages. |
| Outcome: | The proposed methods improve low-resource dependency parsers by transferring syntactic knowledge from one language to another. |