Papers by Aasish Pappu
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts (2024.findings-acl)
Copied to clipboard
Ganesh Jawahar, Haichuan Yang, Yunyang Xiong, Zechun Liu, Dilin Wang, Fei Sun, Meng Li, Aasish Pappu, Barlas Oguz, Muhammad Abdul-Mageed, Laks Lakshmanan, Raghuraman Krishnamoorthi, Vikas Chandra
| Challenge: | Neural architecture search (NAS) uses weight-sharing supernets to generate diverse subnetworks without retraining. |
| Approach: | They propose a weight-sharing supernet that leverages mixture-of-experts to enhance supernet model expressiveness with minimal training overhead. |
| Outcome: | The proposed method achieves state-of-the-art (SoTA) performance in NAS for fast machine translation models, surpassing NAS-BERT and AutoDistil across various model sizes. |
Binary and Ternary Natural Language Generation (2023.acl-long)
Copied to clipboard
| Challenge: | ternary and binary neural networks have proven difficult to optimize since both parameter and output space are discretized . authors demonstrate ternaries and binary models on downstream tasks of summarization and machine translation . |
| Approach: | They propose to use ternary and binary neural networks to optimize for multiplication-free computation . they propose to apply statistics-based quantization for the weights and elastic quantization of the activations to the transformer text generation model. |
| Outcome: | The proposed model outperforms the best existing models on machine translation tasks. |
Unsupervised Neologism Normalization Using Embedding Space Mapping (D19-55)
Copied to clipboard
| Challenge: | Neologisms refer to recent expressions that are specific to certain entities or events, but have not yet been accepted into mainstream language. |
| Approach: | They propose an unsupervised approach for detecting and normalizing neologisms in social media content without relying on parallel training data. |
| Outcome: | The proposed method detects neologisms and normalizes them to canonical words without training data. |
Detecting Extraneous Content in Podcasts (2021.eacl-main)
Copied to clipboard
| Challenge: | Podcast episodes often contain extraneous material interleaved within the audio and the written descriptions . authors present classifiers that leverage both textual and listening patterns to detect such content . |
| Approach: | They propose a classifier that leverages both textual and listening patterns to detect extraneous material in podcast descriptions and audio transcripts. |
| Outcome: | The proposed classifiers improve ROUGE scores and reduce extraneous content in podcast summarization tasks. |
100,000 Podcasts: A Spoken English Document Corpus (2020.coling-main)
Copied to clipboard
Ann Clifton, Sravana Reddy, Yongze Yu, Aasish Pappu, Rezvaneh Rezapour, Hamed Bonab, Maria Eskevich, Gareth Jones, Jussi Karlgren, Ben Carterette, Rosie Jones
| Challenge: | Podcasts are a large and growing repository of spoken audio. |
| Approach: | They propose to use podcasts as a resource for speech processing and linguistics . they use a corpus of 100,000 podcasts to study the complexity of the domain . |
| Outcome: | The Spotify Podcast Dataset is the largest corpus of transcribed speech data . the dataset contains 60,000 hours of podcasts, with a range of genres and styles . |