Papers with PCFGs
Consistent Unsupervised Estimators for Anchored PCFGs (2020.tacl-1)
Copied to clipboard
| Challenge: | a novel approach for learning probabilistic context-free grammars from strings is proposed . strong learning means that there can be many structurally different PCFGs that define the same distribution over strings. |
| Approach: | They propose an algorithm that is a consistent estimator for a class of PCFGs that are anchored . they show that if the grammar is anchored, the parameters can be directly related to distributional properties of the anchoring strings. |
| Outcome: | The proposed algorithm is consistent for a large class of probabilistic context-free grammars . it shows that the proposed algorithm has good finite sample behavior . |
A Generative Model for Lambek Categorial Sequents (2024.lrec-main)
Copied to clipboard
| Challenge: | generative models such as PLC+ generate grammatical sentences with a high probability of being grammatized. |
| Approach: | They propose a generative model, PLC+, for generating Lambek Categorial Grammar(LCG) sequents. |
| Outcome: | The proposed model generates Lambek Categorial Grammar(LCG) sequents and is more robust to probabilistic context-free grammars. |
Generalized chart constraints for efficient PCFG and TAG parsing (P18-2)
Copied to clipboard
| Challenge: | Existing pruning techniques limit chart constraints to PCFGs and cannot be applied to more expressive grammars. |
| Approach: | They propose to apply chart constraints to more expressive grammars and a neural tagger which predicts chart constraints at very high precision. |
| Outcome: | The proposed technique accelerates both PCFG and TAG parsing by two orders of magnitude while improving accuracy. |
Simple Hardware-Efficient PCFGs with Independent Left and Right Productions (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing approaches to scaling dense PCFGs to thousands of nonterminals have shown to be beneficial for unsupervised parsing, but they still perform poorly as a language model and as an unsupervised model. |
| Approach: | They propose a simple PCFG formalism with independent left and right productions that scales more effectively as a language model and as an unsupervised parser. |
| Outcome: | The proposed formalism scales better as a language model and as an unsupervised parser despite imposing a stronger independence assumption compared to low-rank approaches. |
PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work shows that probabilistic context-free grammars with neural parameterization can be effective in unsupervised constituency parsing. |
| Approach: | They propose a parameterization form of PCFGs based on tensor decomposition which has at most quadratic computational complexity in the symbol number. |
| Outcome: | The proposed model improves unsupervised constituency parsing performance across ten languages. |
Categorial grammar induction from raw data (2023.findings-acl)
Copied to clipboard
| Challenge: | a new model for categorial grammar induction is based on raw data without part-of-speech information. |
| Approach: | They propose a grammar induction model that learns from raw data without part-of-speech information. |
| Outcome: | a new model for inducing a basic categorial grammar is developed . the model attains a recall-homogeneity of 0.33 on average, and a bias toward forward function application is added . |
The Limitations of Limited Context for Constituency Parsing (2021.acl-long)
Copied to clipboard
| Challenge: | a language model that is syntax-aware can produce better samples, authors say . a recent study shows that neural approaches to syntax can perform unsupervised syntactic parsing . |
| Approach: | They propose to incorporate syntax into neural approaches in NLP to produce better samples . they find that the first time neural approaches were able to perform unsupervised syntactic parsing . |
| Outcome: | The proposed models can perform unsupervised syntactic parsing, but they are lagging behind . the proposed models are based on a sandbox of probabilistic context-free-grammars . |
Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs (2022.naacl-main)
Copied to clipboard
| Challenge: | Hidden Markov Models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) are widely used structured models. |
| Approach: | They use tensor rank decomposition to reduce computational complexities for a subset of FGGs subsuming HMMs and PCFGs. |
| Outcome: | The proposed model performs better on HMM modeling and unsupervised PCFG parsing than previous work. |
Feature Interactions Reveal Linguistic Structure in Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing features attribution methods for post-hoc interpretability ignore the existence of interactions between the effects of features on the prediction. |
| Approach: | They propose a grey box method to train models to perfection on a formal language classification task using PCFGs. |
| Outcome: | The proposed methods are able to uncover the grammatical rules acquired by the model under specific configurations and provide novel insights into the linguistic structure of the target models. |
Do Transformers Parse while Predicting the Masked Word? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models encode linguistic structures like parse trees while being trained unsupervised. |
| Approach: | They propose to train pre-trained language models to encode linguistic structures like parse trees while unsupervised. |
| Outcome: | The proposed model performs optimally for masked language modeling loss on the English PCFG. |