| Challenge: | prevailing practice in academia evaluates model performance on in-domain evaluation data . however, in many real world applications data on which model is applied may differ from training data - a problem that is not addressed by current literature. |
| Approach: | They propose to use Finnish-OOD out-of-domain treebank for out- of-domain evaluation . they propose to include sections more challenging for the general parser . |
| Outcome: | The proposed treebank includes five distinct data sources and a total of 19,382 syntactic words in 2,122 sentences. |
Similar Papers
Treebank Embedding Vectors for Out-of-Domain Dependency Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | a recent advance in monolingual dependency parsing is the idea of a treebank embedding vector . this allows the model to prefer training data from one treebank over another at test time . |
| Approach: | They propose a method to predict a treebank vector for sentences that do not come from a particular treebank . they also explore what happens when they move away from predefined treebank embedding vectors . |
| Outcome: | The proposed method can predict treebank vectors for sentences that do not come from a treebank used in training with sufficient accuracy for nine out of ten languages. |
Investigating UD Treebanks via Dataset Difficulty Measures (2023.eacl-main)
Copied to clipboard
| Challenge: | Treebanks annotated with Universal Dependencies (UD) are currently available for over 100 languages and are only partially reflected in parser evaluations via accuracy metrics like LAS. |
| Approach: | They propose to use dataset cartography, V-information, and minimum description length to analyze UD treebanks using three accuracy-free methods to provide insights about them. |
| Outcome: | The proposed methods provide insights about UD treebanks that would remain undetected if only LAS was considered. |
Parsing Tweets into Universal Dependencies (N18-1)
Copied to clipboard
| Challenge: | a new tweet treebank for English is designed to analyze tweets with universal dependencies (UD). |
| Approach: | They extend the universal dependencies guidelines to include special constructions in tweets that affect tokenization, part-of-speech tagging, and labeled dependencies. |
| Outcome: | The proposed method outperforms state-of-the-art parsers on other treebanks in accuracy and speed. |
Valency-Augmented Dependency Parsing (D18-1)
Copied to clipboard
| Challenge: | valency analysis is a complex task that requires a large number of subcategorizations, such as the number and types of syntactic dependents. |
| Approach: | They propose a parsing approach that explicitly models the number and types of syntactic dependents as valency patterns and a probabilistic model for tagging them. |
| Outcome: | The proposed approach outperforms the state-of-the-art labeled attachment score on 53 treebanks representing 41 languages and outperformed the previous state- of-the art labeles by 0.7. |
Universal Dependencies According to BERT: Both More Specific and More General (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies show that individual BERT heads encode particular dependency relation types, but they do not match one-to-one. |
| Approach: | They propose a method for relation identification and syntactic tree construction that can be applied with minimal supervision and generalizes well across languages. |
| Outcome: | The proposed method produces significantly more consistent dependency trees than previous work and can be applied with only a minimal amount of supervision and generalizes well across languages. |
Spoken Language Treebanks in Universal Dependencies: an Overview (2022.lrec-1)
Copied to clipboard
| Challenge: | spoken language treebanks have divergent annotation schemes limiting cross-resource explorations . many spoken language trees have no written form, but many of the world languages have no spoken form at all. |
| Approach: | They propose to use the Universal Dependencies annotation scheme to annotate spoken language treebanks using a morphosyntactic annotation scheme. |
| Outcome: | The proposed treebanks differ significantly with respect to the inventory and format of transcribed phenomena and the principles adopted in their morphosyntactic annotation. |
Some Languages Seem Easier to Parse Because Their Treebanks Leak (2020.emnlp-main)
Copied to clipboard
| Challenge: | Cross-language differences in (universal) dependency parsing performance are mostly attributed to treebank size, average sentence length, average dependency length, morphological complexity, and domain differences. |
| Approach: | They compute graph isomorphisms and find that treebank size is a factor that influences parsing performance. |
| Outcome: | The results show that the overlap between training and test graphs explain more of the observed variation than standard explanations such as the above. |
Parser Training with Heterogeneous Treebanks (P18-2)
Copied to clipboard
| Challenge: | In the 2017 CoNLL Shared Task on Universal Dependency Parsing, 25 languages have more than one treebank . many teams did not take advantage of the multiple treebanks, however, and trained one model per treebank instead of one model for each language. |
| Approach: | They propose a method to make the most of heterogeneous treebanks when training a monolingual parser. |
| Outcome: | The proposed method improves on training with multiple treebanks for a single language. |
Universal Dependencies Version 2 for Japanese (L18-1)
Copied to clipboard
Masayuki Asahara, Hiroshi Kanayama, Takaaki Tanaka, Yusuke Miyao, Sumire Uematsu, Shinsuke Mori, Yuji Matsumoto, Mai Omura, Yugo Murawaki
| Challenge: | UD Japanese resources are built on automatic conversion from several treebanks. |
| Approach: | They propose to port the word delimitation, POS, and syntactic relations of existing treebanks to UD Japanese . they discuss the issues of the UD scheme found through porting of the Japanese language . |
| Outcome: | The proposed UD Japanese resources are based on automatic conversion from treebanks. |
The Persian Dependency Treebank Made Universal (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing universal dependency treebanks are lacking sufficient annotated data. |
| Approach: | They propose a method for converting Persian Dependency Treebank to Universal Dependencies using an automatic method. |
| Outcome: | The proposed method is more compatible with Universal Dependencies than the Uppsala Persian Universal Dependency Treebank. |