Papers by Lav Varshney
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding (2020.emnlp-main)
Copied to clipboard
| Challenge: | Inflectional variation is a common feature of World Englishes such as Colloquial Singapore English and African American Vernacular English. |
| Approach: | They propose a method to tokenize English text by reducing inflected words to their base forms before reinjecting the grammatical information as special symbols. |
| Outcome: | The proposed method defends against inflectional adversaries while maintaining performance on clean data. |
A Meta-Learning Perspective on Transformers for Causal Language Modeling (2024.findings-acl)
Copied to clipboard
| Challenge: | Mechanisms of the Transformer architecture for causal language modeling are not well understood. |
| Approach: | They propose a meta-learning view of the Transformer architecture when trained for a causal language modeling task by explicating an inner optimization process that may happen within the Transformer. |
| Outcome: | The proposed model is based on a self-attention mechanism and has been widely used in natural language processing, computer vision, and scientific discovery. |