Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads (2020.aacl-main)
Copied to clipboard
| Challenge: | Existing approaches to analyze syntactic knowledge of pre-trained language models have been limited. |
| Approach: | They propose an unsupervised method that extracts constituency trees from PLM attention heads. |
| Outcome: | The proposed method outperforms existing approaches if no development set is present. |
Similar Papers
Multilingual Chart-based Constituency Parse Extraction from Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for extracting complete (binary) parses from pre-trained language models are expensive and time-consuming. |
| Approach: | They propose a chart-based method and an effective top-K ensemble technique to extractbinary parses from PLMs. |
| Outcome: | The proposed method can induce non-trivial parses for sentences from nine languages in an integrated and language-agnostic manner, and is robust to cross-lingual transfer. |
Unsupervised Parsing via Constituency Tests (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised parsing rely on constituency tests . linguists can judge a sentence's grammatical validity by modifying it via some transformation . |
| Approach: | They propose a method for unsupervised parsing based on a constituency test . they specify a set of transformations and use an unsupervised neural acceptability model to make grammaticality decisions. |
| Outcome: | The proposed method achieves 62.8 F1 on the Penn Treebank test set, an improvement of 7.6 points over the previous best results. |
Revisiting the Practical Effectiveness of Constituency Parse Extraction from Pre-trained Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Constituency Parse Extraction from Pre-trained Language Models (CPE-PLM) is a new paradigm that attempts to induce constituency parse trees based on the internal knowledge of pre-tried language models. |
| Approach: | They propose to use constituency parse trees from pre-trained language models to induce constituency trees by introducing a set of heterogeneous PLMs combined using two advanced ensemble methods. |
| Outcome: | The proposed approach is more effective than typical supervised parsers in few-shot settings. |
Rule Augmented Unsupervised Constituency Parsing (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent studies have shown that unsupervised parsing methods do not learn meaningful semantics (not even simple grammar) |
| Approach: | They propose an approach that utilizes very generic linguistic knowledge of the language present in the form of syntactic grammar rules and is independent of the base system. |
| Outcome: | The proposed model is independent of the base system and takes advantage of syntactic grammar rules. |
An Empirical Comparison of Unsupervised Constituency Parsing Methods (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for unsupervised constituency parsing are inconsistent due to data preprocessing, lexicalization, and evaluation metrics. |
| Approach: | They propose to standardize experimental settings for better comparability between methods . they compare existing methods with those proposed by decade-old models . |
| Outcome: | The proposed methods perform better than decade-old models on English and Japanese, respectively, compared with decade- old models. |
Co-training an Unsupervised Constituency Parser with Weak Supervision (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for unsupervised parsing that use bootstrapping classifiers to identify if a node dominates a span are lacking. |
| Approach: | They propose a method for unsupervised parsing that relies on bootstrapping classifiers to identify if a node dominates a specific span. |
| Outcome: | The proposed method achieves 63.1 F1 on the English test set and new state-of-the-art on treebanks for Chinese and Japanese. |
Unsupervised Natural Language Parsing (Introductory Tutorial) (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | Unsupervised parsing learns a syntactic parser from training sentences without parse tree annotations. |
| Approach: | This tutorial will introduce what unsupervised parsing does and how it can be useful for and beyond syntactic parse. |
| Outcome: | This paper will provide an overview of major approaches to unsupervised parsing and analyze their strengths and weaknesses. |
Rethinking Self-Attention: Towards Interpretability in Neural Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that attention mechanisms provide arguably explainable attention distributions that can help to interpret predictions. |
| Approach: | They propose a new self-attention layer where attention heads represent labels. |
| Outcome: | The proposed model obtains state-of-the-art results on the Penn Treebank and Chinese Treebank. |
Straight to the Tree: Constituency Parsing with Neural Syntactic Distance (P18-1)
Copied to clipboard
| Challenge: | Compared to traditional shift-reduce parsing schemes, our approach is free from the potentially disastrous compounding error. |
| Approach: | They propose a model that predicts a scalar for each split position in a sentence and then determines the topology of grammar tree based on syntactic distances. |
| Outcome: | The proposed model achieves the state-of-the-art single model F1 score of 92.1 on PTB and 86.4 on CTB dataset, surpassing the previous single model results by a large margin. |
Parsing Headed Constituencies (2024.lrec-main)
Copied to clipboard
| Challenge: | Using constituency and dependency trees, syntactic representations are preferred for tasks such as nominal phrase extraction and identification of terminology. |
| Approach: | They propose a parsing technique that generates headed constituency trees which combine information typically contained in constituency and dependency trees. |
| Outcome: | The proposed method generates headed constituency trees with discontinuities and can generate constituency tree with discontinuity. |