Papers by Vladislav Maraev
Can the Transformer Learn Nested Recursion with Symbol Masking? (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on self-attention models show they can generalise to context-free languages . |
| Approach: | They use encoder-only models to train to generalise nested symbols . they find that the predictions made correspond to a simple parenthesis counting strategy . |
| Outcome: | The proposed model can generalise to nested structures at higher nesting depth and with a push-down automaton. |