Papers by Abhishek Panigrahi
Word2Sense: Sparse Interpretable Word Embeddings (P19-1)
Copied to clipboard
| Challenge: | Word2Sense embeddings are interpretable, but they are sparse and fast to compute . a unitary rotation can be applied to many of these embeddables retaining their utility for computational tasks while changing the values of individual coordinates. |
| Approach: | They propose an unsupervised method to generate Word2Sense word embeddings that are interpretable. |
| Outcome: | The proposed method compares well with other unsupervised word embeddings on NLP tasks. |
Do Transformers Parse while Predicting the Masked Word? (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that pre-trained language models encode linguistic structures like parse trees while being trained unsupervised. |
| Approach: | They propose to train pre-trained language models to encode linguistic structures like parse trees while unsupervised. |
| Outcome: | The proposed model performs optimally for masked language modeling loss on the English PCFG. |
Representing Rule-based Chatbots with Transformers (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing work on how Transformers can solve synthetic tasks has not explored how to extend this to a conversational setting. |
| Approach: | They propose to use ELIZA as a framework for formal mechanistic analysis of Transformers . they propose to model local pattern matching and long-term dialogue state tracking . |
| Outcome: | The proposed model can be extended to model key aspects of conversation, the authors show . their model favors an induction head mechanism over a more precise copying mechanism . |