Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | Syntactic parse trees are valuable intermediate features for many NLP pipelines. |
| Approach: | They propose an improved version of DIORA that encodes a single tree rather than a softly-weighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. |
| Outcome: | The proposed model improves state-of-the-art in constituency parsing on the English WSJ Penn Treebank by 2.2-6% F1, depending on the data used for fine-tuning. |
Similar Papers
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders (N19-1)
Copied to clipboard
| Challenge: | Using the deep inside-outside recursive autoencoder, we can extract both shallow parses and full syntactic trees from any domain or language automatically. |
| Approach: | They propose a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. |
| Outcome: | The proposed method outperforms previous methods on the WSJ dataset. |
Deep Inside-outside Recursive Autoencoder with All-span Objective (2020.coling-main)
Copied to clipboard
| Challenge: | Existing neural approaches for constituency parsing are limited for low-resource languages and domains. |
| Approach: | They extend the training objective of DIORA by making use of all spans instead of only leaf-level spans. |
| Outcome: | The proposed model improves on two languages and provides better parsing accuracy than the original model. |
Unsupervised Labeled Parsing with Deep Inside-Outside Recursive Autoencoders (D19-1)
Copied to clipboard
| Challenge: | Existing models that use ground-truth part-of-speech tags are not always available and have significant weaknesses. |
| Approach: | They propose to use deep inside-outside recursive autoencoders to cluster the learned phrase vectors to induce span labels. |
| Outcome: | The proposed model outperforms ELMo and BERT on two versions of the Wall Street Journal dataset and improves over a previous state-of-the-art system that requires additional human annotations by 5 absolute F1 points (19% relative error reduction). |
Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Chart-based models have shown great potential in unsupervised grammar induction, running recursively and hierarchically, but requiring O(n3) time-complexity. |
| Approach: | They propose a model-guided pruning method that scales to large language model pretraining by introducing a heuristic pruning method. |
| Outcome: | The proposed method significantly improves grammar induction quality and achieves competitive results in downstream tasks. |
Rule Augmented Unsupervised Constituency Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that unsupervised parsing methods do not learn meaningful semantics (not even simple grammar) |
| Approach: | They propose an approach that utilizes very generic linguistic knowledge of the language present in the form of syntactic grammar rules and is independent of the base system. |
| Outcome: | The proposed model is independent of the base system and takes advantage of syntactic grammar rules. |
Efficient Second-Order TreeCRF for Neural Dependency Parsing (2020.acl-main)
Copied to clipboard
| Challenge: | In the deep learning (DL) era, dependency parsing models are extremely simplified with little hurt on performance thanks to the remarkable capability of multi-layer BiLSTMs in context representation. |
| Approach: | They propose to extend the biaffine parser to a second-order TreeCRF extension to reduce the complexity of the inside-outside algorithm. |
| Outcome: | The proposed extension can be used to batchify the inside and Viterbi algorithms and avoid the complex outside algorithm via efficient back-propagation. |
Please Mind the Root: Decoding Arborescences for Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dependency tree has a root constraint, but only one edge may emanate from the root node. |
| Approach: | They propose an algorithm which enforces a root constraint without compromising the original runtime. |
| Outcome: | The proposed algorithm satisfies the constraint without compromising the original runtime. |
Co-training an Unsupervised Constituency Parser with Weak Supervision (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for unsupervised parsing that use bootstrapping classifiers to identify if a node dominates a span are lacking. |
| Approach: | They propose a method for unsupervised parsing that relies on bootstrapping classifiers to identify if a node dominates a specific span. |
| Outcome: | The proposed method achieves 63.1 F1 on the English test set and new state-of-the-art on treebanks for Chinese and Japanese. |
A Root of a Problem: Optimizing Single-Root Dependency Parsing (2021.emnlp-main)
Copied to clipboard
| Challenge: | Graph-based dependency parsers can be improved without compromising on accuracy or accuracy. |
| Approach: | They propose two approaches to single-root dependency parsing that yield speed ups . they show that one approach is fully correct and finds the optimal dependency tree . |
| Outcome: | The proposed approach finds the optimal dependency tree without loss of accuracy or optimality. |
Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories (2021.tacl-1)
Copied to clipboard
| Challenge: | Combinatory Categorial Grammar (CCG) parsers operate as a pipeline with a large search space of complex 'supertags' . |
| Approach: | They propose to use CCG supertags to generate CCG categories that have never been seen in training and to use tree-structured prediction to account for their internal structure. |
| Outcome: | The proposed model recovers a fraction of the long-tail supertags while approximating the state of the art in overall tag accuracy with fewer parameters. |