Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders (N19-1)
Copied to clipboard
| Challenge: | Using the deep inside-outside recursive autoencoder, we can extract both shallow parses and full syntactic trees from any domain or language automatically. |
| Approach: | They propose a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. |
| Outcome: | The proposed method outperforms previous methods on the WSJ dataset. |
Similar Papers
Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders (2020.emnlp-main)
Copied to clipboard
| Challenge: | Syntactic parse trees are valuable intermediate features for many NLP pipelines. |
| Approach: | They propose an improved version of DIORA that encodes a single tree rather than a softly-weighted mixture of trees by employing a hard argmax operation and a beam at each cell in the chart. |
| Outcome: | The proposed model improves state-of-the-art in constituency parsing on the English WSJ Penn Treebank by 2.2-6% F1, depending on the data used for fine-tuning. |
Deep Inside-outside Recursive Autoencoder with All-span Objective (2020.coling-main)
Copied to clipboard
| Challenge: | Existing neural approaches for constituency parsing are limited for low-resource languages and domains. |
| Approach: | They extend the training objective of DIORA by making use of all spans instead of only leaf-level spans. |
| Outcome: | The proposed model improves on two languages and provides better parsing accuracy than the original model. |
Unsupervised Labeled Parsing with Deep Inside-Outside Recursive Autoencoders (D19-1)
Copied to clipboard
| Challenge: | Existing models that use ground-truth part-of-speech tags are not always available and have significant weaknesses. |
| Approach: | They propose to use deep inside-outside recursive autoencoders to cluster the learned phrase vectors to induce span labels. |
| Outcome: | The proposed model outperforms ELMo and BERT on two versions of the Wall Street Journal dataset and improves over a previous state-of-the-art system that requires additional human annotations by 5 absolute F1 points (19% relative error reduction). |
Fast-R2D2: A Pretrained Recursive Neural Network based on Pruned CKY for Grammar Induction and Text Representation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Chart-based models have shown great potential in unsupervised grammar induction, running recursively and hierarchically, but requiring O(n3) time-complexity. |
| Approach: | They propose a model-guided pruning method that scales to large language model pretraining by introducing a heuristic pruning method. |
| Outcome: | The proposed method significantly improves grammar induction quality and achieves competitive results in downstream tasks. |
Rule Augmented Unsupervised Constituency Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that unsupervised parsing methods do not learn meaningful semantics (not even simple grammar) |
| Approach: | They propose an approach that utilizes very generic linguistic knowledge of the language present in the form of syntactic grammar rules and is independent of the base system. |
| Outcome: | The proposed model is independent of the base system and takes advantage of syntactic grammar rules. |
On Eliciting Syntax from Language Models via Hashing (2024.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised parsing aims to infer syntactic structure from raw text . despite its importance, advancements in this task have been slow . |
| Approach: | They propose to use unsupervised parsing to infer syntactic structure from raw text . they upgrade the bit-level CKY to first-order to encode lexicon and syntax . |
| Outcome: | The proposed method shows competitive performance on various datasets. |
Improved Latent Tree Induction with Distant Supervision via Span Constraints (2021.emnlp-main)
Copied to clipboard
Zhiyang Xu, Andrew Drozdov, Jay Yoon Lee, Tim O’Gorman, Subendhu Rongali, Dylan Finkbeiner, Shilpa Suresh, Mohit Iyyer, Andrew McCallum
| Challenge: | Distant supervision is not a practical way to perform unsupervised syntactic parsing. |
| Approach: | They propose a technique that uses distant supervision to improve unsupervised constituency parsing by using phrase bracketing. |
| Outcome: | The proposed method improves constituency parsing on English WSJ Penn Treebank by more than 5 F1 compared with full parse tree annotations. |
Recursive Top-Down Production for Sentence Generation with Latent Trees (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Various studies have shown that incorporating syntactic structures into recursive encoders can be beneficial for various natural language tasks. |
| Approach: | They propose a dynamic programming algorithm that marginalises over latent binary tree structures with N leaves to train a recursive neural function. |
| Outcome: | The proposed model outperforms previous models on the LENGTH split and English question formation tasks on the Multi30k dataset. |
End-to-End Reinforcement Learning for Automatic Taxonomy Induction (P18-1)
Copied to clipboard
| Challenge: | Existing methods for automating taxonomy induction often divide the problem into two subtasks . a novel end-to-end reinforcement learning approach is proposed to improve the accuracy of such methods. |
| Approach: | They propose an end-to-end reinforcement learning approach to automatic taxonomy induction from a set of terms. |
| Outcome: | The proposed approach outperforms state-of-the-art methods on two public datasets of different domains. |
Co-training an Unsupervised Constituency Parser with Weak Supervision (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for unsupervised parsing that use bootstrapping classifiers to identify if a node dominates a span are lacking. |
| Approach: | They propose a method for unsupervised parsing that relies on bootstrapping classifiers to identify if a node dominates a specific span. |
| Outcome: | The proposed method achieves 63.1 F1 on the English test set and new state-of-the-art on treebanks for Chinese and Japanese. |