Papers by Ayush Kaushal
Efficient Encoders for Streaming Sequence Tagging (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing bidirectional encoders require a restart when a new token is received. |
| Approach: | They propose a Hybrid Encoder with Adaptive Restart that enables asynchronous encoding of a new token in an incremental streaming input. |
| Outcome: | The proposed encoder offers FLOP savings in streaming settings up to 71.1% and outperforms bidirectional encoders for streaming predictions by up to +0% streaming exact match. |
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP (2021.emnlp-main)
Copied to clipboard
Zhijing Jin, Julius von Kügelgen, Jingwei Ni, Tejas Vaidhya, Ayush Kaushal, Mrinmaya Sachan, Bernhard Schoelkopf
| Challenge: | a meta-analysis of published studies shows that the causal direction of data collection can explain some trends in NLP . semi-supervised learning and domain adaptation performance differ on a number of tasks . |
| Approach: | They argue that the causal direction of the data collection process has nontrivial implications . authors categorize common NLP tasks according to their causal direction . they also empirically assay the validity of the ICM principle for text data . |
| Outcome: | The proposed model can explain differences in semi-supervised learning and domain adaptation performance across settings. |
What do tokens know about their characters and how do they know it? (2022.naacl-main)
Copied to clipboard
| Challenge: | Pre-trained language models that use subword tokenization schemes can succeed at a variety of language tasks that require character-level information. |
| Approach: | They propose to use word tokenization schemes to probe what word pieces encode . they show that larger models can encode character-level information . |
| Outcome: | The proposed models can encode character-level information and perform better on non-Latin alphabets. |
tWT–WT: A Dataset to Assert the Role of Target Entities for Detecting Stance of Tweets (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing stance detection datasets are complex deep neural networks, making them difficult to interpret. |
| Approach: | They propose a new large dataset free of such biases and demonstrate its aptness on existing stance detection systems. |
| Outcome: | The proposed model achieves human-level performance on the WT–WT dataset and more than two-third accuracy on other datasets. |
Scaling Laws and Efficient Inference for Ternary Language Models (2025.acl-long)
Copied to clipboard
Tejas Vaidhya, Ayush Kaushal, Vineet Jain, Francis Couture-Harpin, Prashant Shishodia, Majid Behbahani, Yuriy Nevmyvaka, Irina Rish
| Challenge: | Large language models (LLMs) are increasingly used across research and industry applications, yet their inference efficiency remains a challenge. |
| Approach: | They propose ternary language models that employ quantization-aware training to significantly reduce memory requirements. |
| Outcome: | The proposed ternary language models demonstrate sustained performance gains at scale. |