Papers by Ekin Akyürek
Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks (2024.naacl-long)
Copied to clipboard
Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, Yoon Kim
| Challenge: | Recent language models possess impressive performance across a wide range of tasks . however, they often rely on narrow, non-transferable procedures for task-solving . |
| Approach: | They propose to evaluate language models using "counterfactual" task variants that deviate from standard tasks. |
| Outcome: | The proposed framework shows that language models perform better on a wide range of tasks compared to the default conditions. |
Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing language models (LMs) generate factually correct text and estimate truth values of individual claims, but they do not reflect a coherent, manipulable model of the world. |
| Approach: | They propose a method that uses language models to identify implications of (and contradictions within) the text they generate. |
| Outcome: | The proposed method improves LM factuality by 3-26% across the CREAK, MQuAKE, and Reversal Curse datasets. |