Papers by Shantanu Acharya
SWAN: An Efficient and Scalable Approach for Long-Context Language Modeling (2025.emnlp-main)
Copied to clipboard
Krishna C Puvvada, Faisal Ladhak, Santiago Akle Serano, Cheng-Ping Hsieh, Shantanu Acharya, Somshubra Majumdar, Fei Jia, Samuel Kriman, Simeng Sun, Dima Rekesh, Boris Ginsburg
| Challenge: | Existing decoder-only models struggle with context lengths beyond their training distribution. |
| Approach: | They propose a causal Transformer architecture that generalizes robustly to sequence lengths longer than seen during training. |
| Outcome: | The proposed decoder-only architecture can generalize robustly to longer contexts . it is more computationally efficient than the standard Transformer architecture, the authors say . |
Every Child Should Have Parents: A Taxonomy Refinement Algorithm Based on Hyperbolic Term Embeddings (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to domain-specific taxonomy induction from text are relying on distributional semantics for hyponym-hypernym relationships, but many of them learn prototypical hypernymes, not taking into account the relation between both terms in classification. |
| Approach: | They propose to use Poincaré embeddings to improve existing approaches to domain-specific taxonomy induction from text as a signal for relocating wrong hyponym terms and attaching disconnected terms in a taxonomies. |
| Outcome: | The proposed method significantly improves state-of-the-art methods on the SemEval-2016 Task 13 on taxonomy extraction. |