Papers by Randall O’Reilly
Compositional Generalization by Factorizing Alignment and Translation (2020.acl-srw)
Copied to clipboard
| Challenge: | a crucial property underlying the expressive power of human language is its systematicity. |
| Approach: | They propose to make an analogous separation between alignment and translation in neural machine translation to capture compositional structure. |
| Outcome: | The proposed architecture outperforms existing neural networks on a compositional generalization task without supervision. |
Systematicity Emerges in Transformers when Abstract Grammatical Roles Guide Attention (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing systems that use transformers lack systematicity, but they are inferior to human learners in sample efficiency and difficult generalization problems. |
| Approach: | They propose to modify a transformer so that it controls attention distributions and fills in the gaps. |
| Outcome: | The proposed model shows that the performance of natural language processing systems is improved when abstract role labels are assigned to the input stream and provided to the role stream. |