Papers by David Wingate
Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models (2022.findings-emnlp)
Copied to clipboard
| Challenge: | We explore the idea of compressing the prompts used to condition language models. |
| Approach: | They explore the idea of compressing the prompts used to condition language models . they show that compressed prompts can retain a substantive amount of information about the original prompt . |
| Outcome: | The proposed method can be extended to controllability and toxicity reduction. |
Features that Make a Difference: Leveraging Gradients for Improved Dictionary Learning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Sparse Autoencoders (SAEs) are a promising approach for extracting neural network representations by learning a sparse and overcomplete decomposition of the network’s internal activations. |
| Approach: | They propose a method that learns a sparse and overcomplete decomposition of the network's internal activations and a gradient approach to learn latents. |
| Outcome: | The proposed algorithms improve the performance of the k-sparse autoencoder and the ability to learn latent features. |
An Information-theoretic Approach to Prompt Engineering Without Ground Truth Labels (2022.acl-long)
Copied to clipboard
Taylor Sorensen, Joshua Robinson, Christopher Rytting, Alexander Shaw, Kyle Rogers, Alexia Delorey, Mahmoud Khalil, Nancy Fulda, David Wingate
| Challenge: | Existing prompt engineering methods require labeled data and access to model parameters . a new method for selecting prompt templates without labeles and without direct access to the model is needed. |
| Approach: | They propose a method for selecting prompt templates without labeled examples and without direct access to the model. |
| Outcome: | The proposed method performs at almost oracle levels, without labels, on 7 datasets representing 7 different NLP tasks. |