Structural generalization in COGS: Supertagging is (almost) all you need (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies have shown that neural networks fail to generalize on out-of-distribution examples. |
| Approach: | They extend a neural graph-based parsing framework to address compositional generalization limitations . they introduce a supertagging step with valency constraints and reduce the graph prediction problem . |
| Outcome: | The proposed approach improves results on COGS datasets that require structural generalization. |
Similar Papers
Improving Compositional Generalization in Classification Tasks via Structure Annotations (2021.acl-short)
Copied to clipboard
| Challenge: | Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. |
| Approach: | They propose to convert a natural language sequence-to-sequence dataset into a classification dataset that requires compositional generalization. |
| Outcome: | The proposed model can generalize compositionally by providing hints on the structure of the input. |
Generalized chart constraints for efficient PCFG and TAG parsing (P18-2)
Copied to clipboard
| Challenge: | Existing pruning techniques limit chart constraints to PCFGs and cannot be applied to more expressive grammars. |
| Approach: | They propose to apply chart constraints to more expressive grammars and a neural tagger which predicts chart constraints at very high precision. |
| Outcome: | The proposed technique accelerates both PCFG and TAG parsing by two orders of magnitude while improving accuracy. |
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language is characterized by compositionality: meaning of complex expressions is constructed from the meanings of its constituent parts. |
| Approach: | They propose a semantic parsing dataset based on a fragment of English to assess compositional generalization abilities. |
| Outcome: | The proposed model can generalize meanings in a given sentence in 96–99% of the tests, but generalization accuracy is lower and the generalization sensitivity is higher. |
Structural generalization is hard for sequence-to-sequence models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Sequence-to-sequence models have been successful across many NLP tasks, but they have low generalization accuracy . |
| Approach: | They propose to use linguistic knowledge to overcome generalization limitations of seq2seq models . they show that human beings are able to understand and produce linguistic structures they have never observed before . |
| Outcome: | The proposed models can overcome this limitation by having linguistic knowledge built in. |
SLOG: A Structural Generalization Benchmark for Semantic Parsing (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing compositional generalization benchmarks focus on lexical generalisation, the interpretation of novel lexicals in syntactic structures familiar from training. |
| Approach: | They propose a semantic parsing dataset that extends COGS with 17 structural generalization cases to evaluate how well models generalize to new complex linguistic expressions. |
| Outcome: | The proposed model generalization accuracy is far below the near-perfect accuracy of existing models on COGS, demonstrating the role of SLOG in foregrounding the large discrepancy between models’ lexical and structural generalization capacities. |
Cross-Domain Generalization of Neural Constituency Parsers (P19-1)
Copied to clipboard
| Challenge: | Neural parsers perform well on in-domain benchmarks, but their performance degrades in well-understood ways. |
| Approach: | They analyze generalization on English and Chinese corpora to see if they can generalize to other domains. |
| Outcome: | The proposed neural parsers perform better on in-domain benchmarks than on out-of-domain corpora. |
Compositional Generalization without Trees using Multiset Tagging and Latent Permutations (2023.acl-long)
Copied to clipboard
| Challenge: | Seq2seq models struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions or deeper recursion of phenomena that the model handles correctly in isolation. |
| Approach: | They propose a new way of parameterizing and predicting permutations by combining input tokens with multisets of output tokens and a method to backpropagate through the solver. |
| Outcome: | The proposed model outperforms pretrained models and prior work on realistic semantic parsing tasks that require generalization to longer examples. |
Compositional Generalization by Factorizing Alignment and Translation (2020.acl-srw)
Copied to clipboard
| Challenge: | a crucial property underlying the expressive power of human language is its systematicity. |
| Approach: | They propose to make an analogous separation between alignment and translation in neural machine translation to capture compositional structure. |
| Outcome: | The proposed architecture outperforms existing neural networks on a compositional generalization task without supervision. |
Compositional Generalization via Semantic Tagging (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural sequence-to-sequence models fail at compositional generalization, i.e., they cannot generalize to unseen compositions of seen components. |
| Approach: | They propose a decoding framework that preserves expressivity and generality of sequence-to-sequence models while featuring lexicon-style alignments and disentangled information processing. |
| Outcome: | The proposed framework improves compositional generalization across model architectures, domains, and semantic formalisms on three semantic parsing datasets. |
End-to-End Graph-Based TAG Parsing with Neural Networks (N18-1)
Copied to clipboard
| Challenge: | Using BiLSTMs, highway connections, and character-level CNNs, we propose a graph-based Tree Adjoining Grammar (TAG) parser. |
| Approach: | They propose a graph-based Tree Adjoining Grammar parser that uses BiLSTMs, highway connections, and character-level CNNs. |
| Outcome: | The proposed parser outperforms the previously reported best by more than 2.2 LAS and UAS points. |