Papers by Ekin Akyurek
LexSym: Compositionality as Lexical Symmetry (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to generalize compositional models fail to generalise from small datasets. |
| Approach: | They propose a domain-general and model-agnostic formulation of compositionality as a constraint on symmetries of data distributions rather than models. |
| Outcome: | The proposed procedure matches or surpasses state-of-the-art, task-specific models on COGS semantic parsing, SCAN and Alchemy instruction following, and CLEVR-CoGenT visual question answering datasets. |
Lexicon Learning for Few Shot Sequence Modeling (2021.acl-long)
Copied to clipboard
| Challenge: | Past work has shown that many failures of systematic generalization arise from neural models’ inability to disentangle lexical phenomena from syntactic ones. |
| Approach: | They propose a lexical translation mechanism that generalizes existing copy mechanisms to incorporate learned, decontextualized, token-level translation rules. |
| Outcome: | The proposed model improves generalization on a diverse set of sequence modeling tasks drawn from cognitive science, formal semantics, and machine translation. |
RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs (2023.acl-long)
Copied to clipboard
| Challenge: | Despite their success, even the largest language models make mistakes. |
| Approach: | They propose a framework where one language model can generate critiques to improve its peer's performance. |
| Outcome: | The proposed framework improves the performance of a fixed model 200 times its size by 10% over other models. |
Towards Tracing Knowledge in Language Models Back to the Training Data (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work on training data attribution (TDA) may offer effective tools for identifying such examples, known as "proponents". |
| Approach: | They propose a benchmark to identify which training examples taught an LM to generate a particular factual assertion. |
| Outcome: | The proposed methods have lower proponent-retrieval precision than baselines that do not have access to the LM. |