Papers with SoVQ
Inducing Systematicity in Transformers by Attending to Structurally Quantized Embeddings (2024.acl-long)
Copied to clipboard
| Challenge: | Empirically, we show SQ-Transformer achieves stronger compositional generalization than the vanilla Transformer on low-complexity datasets. |
| Approach: | They propose a Transformer that explicitly encourages systematicity in the embeddings and attention layers even with low-complexity data. |
| Outcome: | Empirically, the proposed model achieves stronger compositional generalization than the vanilla Transformer on low-complexity datasets. |