Real-World Compositional Generalization with Disentangled Sequence-to-Sequence Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to compositional generalization have been designed with semantic parsing in mind. |
| Approach: | They propose a disentangled sequence-to-sequence model which encourages more disentanglement and improves its compute and memory efficiency. |
| Outcome: | The proposed model improves generalization performance across existing tasks and datasets and a new machine translation benchmark. |
Similar Papers
Disentangled Sequence to Sequence Learning for Compositional Generalization (2022.acl-long)
Copied to clipboard
| Challenge: | Existing models struggle to generalize to unseen compositions of seen components . a new approach allows for disentangled representations and better generalization . |
| Approach: | They propose an extension to sequence-to-sequence models which encourage disentanglement by re-encoding source input. |
| Outcome: | The proposed extension delivers better generalization and more disentangled representations . human expressions can be understood by combining known atomic components . |
Compositional Generalization via Semantic Tagging (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing neural sequence-to-sequence models fail at compositional generalization, i.e., they cannot generalize to unseen compositions of seen components. |
| Approach: | They propose a decoding framework that preserves expressivity and generality of sequence-to-sequence models while featuring lexicon-style alignments and disentangled information processing. |
| Outcome: | The proposed framework improves compositional generalization across model architectures, domains, and semantic formalisms on three semantic parsing datasets. |
Improving Compositional Generalization in Classification Tasks via Structure Annotations (2021.acl-short)
Copied to clipboard
| Challenge: | Compositional generalization is the ability to generalize systematically to a new data distribution by combining known components. |
| Approach: | They propose to convert a natural language sequence-to-sequence dataset into a classification dataset that requires compositional generalization. |
| Outcome: | The proposed model can generalize compositionally by providing hints on the structure of the input. |
Learning to Compose Representations of Different Encoder Layers towards Improving Compositional Generalization (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies show that sequence-to-sequence (seq2sequ) models struggle with compositional generalization (CG) a crucial property of human language learning is its compositional globalization (GC), the algebraic ability to understand and produce a potentially infinite number of novel combinations from known components. |
| Approach: | They propose a sequence-to-sequence (seq2sequ) extension which learns to compose representations of different encoder layers dynamically for different tasks. |
| Outcome: | The proposed model achieves competitive results on two comprehensive and realistic benchmarks, which empirically demonstrates the effectiveness of the proposed model. |
Revisiting the Compositional Generalization Abilities of Neural Sequence Models (2022.acl-short)
Copied to clipboard
| Challenge: | Existing studies have suggested that standard seq-to-seq models lack the ability to generalize compositionally. |
| Approach: | They propose to use one-shot primitive generalization as introduced by the popular SCAN benchmark to modify the training distribution in simple and intuitive ways to achieve near-perfect generalization performance. |
| Outcome: | The proposed model achieves near-perfect generalization performance despite a lack of training data . |
Compositional Generalization by Factorizing Alignment and Translation (2020.acl-srw)
Copied to clipboard
| Challenge: | a crucial property underlying the expressive power of human language is its systematicity. |
| Approach: | They propose to make an analogous separation between alignment and translation in neural machine translation to capture compositional structure. |
| Outcome: | The proposed architecture outperforms existing neural networks on a compositional generalization task without supervision. |
Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both? (2021.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to semantic parsing only evaluated on synthetic datasets that are not representative of natural language variation. |
| Approach: | They propose a semantic parsing approach that handles both natural language variation and compositional generalization. |
| Outcome: | The proposed model outperforms existing models across compositional generalization challenges on non-synthetic datasets while being competitive with the state-of-the-art on standard evaluations. |
Grammar-based Decoding for Improved Compositional Generalization in Semantic Parsing (2023.findings-acl)
Copied to clipboard
| Challenge: | Sequence-to-sequence (seq2sequ) models have been successful in semantic parsing tasks but struggle on out-of-distribution data. |
| Approach: | They propose to use a large-scale dialogue dataset to evaluate compositional generalization of semantic parsing. |
| Outcome: | The proposed model outperforms BART- and T5-based models on the SMCalflow-CS dataset on the zero-shot learning task. |
Learning Disentangled Representations for Natural Language Definitions (2023.findings-eacl)
Copied to clipboard
| Challenge: | Disentangling the encodings of neural models is a fundamental aspect for improving interpretability, semantic control and downstream task performance in Natural Language Processing. |
| Approach: | They propose to use syntactic and semantic regularities in textual data to provide models with both structural biases and generative factors. |
| Outcome: | The proposed model outperforms baselines on several qualitative and quantitative benchmarks and improves the results in the downstream task of definition modeling. |
Compositional Generalization for Data-to-Text Generation (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Data-to-text generation models can be used to generate textual descriptions from structured data . despite advances, systems struggle when confronted with unseen combinations of predicates . |
| Approach: | They propose a data-to-text generation model that addresses compositional generalization by clustering predicates into groups. |
| Outcome: | The proposed model outperforms T5-baselines in all evaluation metrics. |