Simple and effective data augmentation for compositional generalization (2024.naacl-long)
Copied to clipboard
| Challenge: | Compositional generalization is the ability of a system to correctly predict the meaning of complex sentences when trained on simpler sentences. |
| Approach: | They propose to use data augmentation methods to generate additional training data by sampling from an augmentation distribution to generalize to the out-of-distribution test data. |
| Outcome: | The proposed method outperforms existing methods that sampled from the training distribution and outperformed existing methods. |
Similar Papers
Align and Augment: Generative Data Augmentation for Compositional Generalization (2024.eacl-long)
Copied to clipboard
| Challenge: | Recent work on semantic parsing has shown that seq2seq models find compositional generalization challenging. |
| Approach: | They propose a data-augmentation strategy that exploits alignment annotations between sentences and their corresponding meaning representations to improve compositional generalization. |
| Outcome: | The proposed model improves compositional generalization performance by exploiting alignment annotations between sentences and their corresponding meaning representations. |
On Evaluating Multilingual Compositional Generalization with Translated Datasets (2023.acl-long)
Copied to clipboard
| Challenge: | a growing amount of research investigating compositional generalization in NLP is done on English . a critical semantic distortion is a limitation of the translation of datasets . |
| Approach: | They propose to translate a dataset for evaluating compositional generalization in semantic parsing. |
| Outcome: | The proposed benchmarks show that the translation of the MCWQ dataset suffers from semantic distortion. |
Data Factors for Better Compositional Generalization (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent diagnostic datasets on compositional generalization expose severe problems . state-of-the-art models trained on larger and more general datasets show better generalization ability . |
| Approach: | They conduct an empirical analysis by training Transformer models on a variety of training sets with different data factors including dataset scale, pattern complexity, example difficulty, etc. |
| Outcome: | The proposed model training on larger datasets improves on compositional generalization tasks. |
Learning to Substitute Spans towards Improving Compositional Generalization (2023.acl-long)
Copied to clipboard
| Challenge: | despite the rising prevalence of neural sequence models, there is a deficiency in compositional generalization. |
| Approach: | They propose a compositional augmentation strategy that enables multi-grained composition of substructures in the whole training set. |
| Outcome: | The proposed strategy outperforms existing strategies on three compositional generalization benchmarks. |
Improving Compositional Generalization in Classification Tasks via Structure Annotations (2021.acl-short)
Copied to clipboard
| Challenge: | Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. |
| Approach: | They propose to convert a natural language sequence-to-sequence dataset into a classification dataset that requires compositional generalization. |
| Outcome: | The proposed model can generalize compositionally by providing hints on the structure of the input. |
Revisiting the Compositional Generalization Abilities of Neural Sequence Models (2022.acl-short)
Copied to clipboard
| Challenge: | Existing studies have suggested that standard seq-to-seq models lack the ability to generalize compositionally. |
| Approach: | They propose to use one-shot primitive generalization as introduced by the popular SCAN benchmark to modify the training distribution in simple and intuitive ways to achieve near-perfect generalization performance. |
| Outcome: | The proposed model achieves near-perfect generalization performance despite a lack of training data . |
Improving Compositional Generalization with Self-Training for Data-to-Text Generation (2022.acl-long)
Copied to clipboard
| Challenge: | Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs). |
| Approach: | They propose a template-based input representation that greatly improves the model’s generalization capability. |
| Outcome: | The proposed model improves tree accuracy by 46%+ and reduces slot error rates by 73%+ over the strong baselines on SGD and Weather benchmarks. |
Quantifying Compositionality of Classic and State-of-the-Art Embeddings (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Static word embeddings make strong claims about compositionality, but the SOTA generative models go too far in the other direction. |
| Approach: | a new study evaluates the compositionality of word embeddings by canonical correlation analysis . strong compositional signals are observed in later training stages across data modalities . |
| Outcome: | a new evaluation of compositional models shows that they exploit access meanings when justified . strong compositional signals are observed in later training stages and in deeper layers of the transformer-based model before a decline at the top layer. |
Improving Compositional Generalization with Latent Structure and Data Augmentation (2022.naacl-main)
Copied to clipboard
| Challenge: | Generic unstructured neural networks struggle on out-of-distribution compositional generalization. |
| Approach: | They propose a method to recombinate examples from a model called Compositional Structure Learner and add them to a pre-trained sequence-to-sequence model. |
| Outcome: | The proposed model is even stronger than a T5-CSL ensemble on two real world compositional generalization tasks. |
Meta-Learning to Compositionally Generalize (2021.acl-long)
Copied to clipboard
| Challenge: | Existing studies show that neural networks struggle with compositional generalization . prior work asserts that there are fundamental differences between cognitive and connectionist architectures that make compositional globalization unlikely. |
| Approach: | They propose a meta-learning augmented version of supervised learning that optimizes for out-of-distribution generalization. |
| Outcome: | The proposed model improves generalization performance on COGS and SCAN datasets. |