Papers by Emily Reif
Data Similarity is Not Enough to Explain Language Model Performance (2023.emnlp-main)
Copied to clipboard
| Challenge: | Large language models perform well on many but not all downstream tasks. |
| Approach: | They compare large language models with downstream benchmarks to determine whether similarity measures correlate with model performance. |
| Outcome: | The results show that similarity measures are not correlated with accuracy or each other in other benchmarks. |
The Case for a Single Model that can Both Generate Continuations and Fill-in-the-Blank (2022.findings-naacl)
Copied to clipboard
| Challenge: | a natural language generation system can be used to create text at the end of a passage . fill in the blank (FITB) is a task of inserting text into a specified position in a text . |
| Approach: | They evaluate the feasibility of using a single model to perform both tasks . they show that models pre-trained with a FitB-style objective are capable of both tasks. |
| Outcome: | The proposed model can perform both fill in the blank and continuation tasks. |
A Pretrainer’s Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity (2024.naacl-long)
Copied to clipboard
Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, Daphne Ippolito
| Challenge: | a large number of pretraining data design practices are under-documented, authors say . authors: strong performance of modern language models depends on selfsupervised pretraining . |
| Approach: | They propose to pretrain models on data curated at different collection times . they find temporal shift between evaluation data and pretraining data leads to performance degradation . |
| Outcome: | The results validate, quantify, and expose many undocumented intuitions about text pretraining . authors say this practice has outperformed other models in the field . |
A Recipe for Arbitrary Text Style Transfer with Large Language Models (2022.acl-short)
Copied to clipboard
| Challenge: | augmented zero-shot learning is a prompting method that allows large language models to perform zero-shoot text style transfer to arbitrary styles, without any model fine-tuning or exemplars in the target style. |
| Approach: | They propose a prompting method that frames style transfer as a sentence rewriting task and requires only a natural language instruction. |
| Outcome: | The proposed method is based on a large language model and is shown to perform on standard style transfer tasks and arbitrary transformations. |
The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Models (2020.emnlp-demos)
Copied to clipboard
Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, Ann Yuan
| Challenge: | Existing tools for modeling and understanding models are limited . existing tools can assist practitioners in understanding and evaluating models . |
| Approach: | They present an open-source platform for visualization and understanding of NLP models. |
| Outcome: | The language interpretability tool (lit) is an open-source platform for visualization and understanding of NLP models. |