Papers by Nikhil Rao
Regularized Graph Convolutional Networks for Short Text Classification (2020.coling-industry)
Copied to clipboard
| Challenge: | Short text classification is a problem in natural language processing, social network analysis, and e-commerce. |
| Approach: | They propose a short text classification technique that incorporates label dependencies into the output space to overcome the limitations of short text. |
| Outcome: | The proposed model outperforms baseline methods on proprietary and external datasets and is more robust to noise in textual features. |
Automatic Pair Construction for Contrastive Post-training (2024.findings-naacl)
Copied to clipboard
Canwen Xu, Corby Rosset, Ethan Chau, Luciano Corro, Shweti Mahajan, Julian McAuley, Jennifer Neville, Ahmed Awadallah, Nikhil Rao
| Challenge: | Large language models (LLMs) have unprecedented proficiency in a wide array of tasks. |
| Approach: | They propose a way to construct contrastive data using preference pairs from multiple models of varying strengths using SLiC and DPO. |
| Outcome: | The proposed method outperforms existing models like Orca in the comparison of SLiC and DPO with SFT baselines. |
Dodo: Dynamic Contextual Compression for Decoder-only LMs (2024.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to NLP are sparsifying attention patterns or approximating the attention computation with kernel methods. |
| Approach: | They propose a method for dynamic contextual compression for decoder-only LMs. |
| Outcome: | The proposed method reduces the cost of self-attention to a fraction of typical time and space. |
Learning Robust Models for e-Commerce Product Search (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models that understand search intent are difficult to learn due to lack of labeled datasets. |
| Approach: | They develop a deep, end-to-end model that learns to effectively classify mismatches . they introduce a latent variable into the cross-entropy loss that alternates between real and generated samples . |
| Outcome: | The proposed model achieves a relative gain of over 26% in F-score and 17% in Area Under PR curve on live search traffic in multiple countries. |