Papers by Harshil Vejendla
SliceMoE: Routing Embedding Slices Instead of Tokens for Fine-Grained and Balanced Transformer Scaling (2025.emnlp-main)
Copied to clipboard
| Challenge: | Token-level routing assigns an entire semantic spectrum to each expert, creating capacity bottlenecks, load-balancing pathologies, and limited specialisation. |
| Approach: | They propose an architecture that routes contiguous slices of a token’s hidden vector and a lightweight shared router predicts the top-k experts. |
| Outcome: | The proposed architecture achieves 1.7x faster inference than dense baselines, 12–18% lower perplexity than parameter-matched token-MoE, and improved expert balance. |
Drift-Adapter: A Practical Approach to Near Zero-Downtime Embedding Model Upgrades in Vector Databases (2025.emnlp-main)
Copied to clipboard
| Challenge: | Upgrading embedding models in production environments requires re-encoding the entire corpus and rebuilding the Approximate Nearest Neighbor (ANN) index. |
| Approach: | They propose a lightweight, learnable transformation layer designed to bridge embedding spaces between models by mapping new queries into the legacy embeddable space. |
| Outcome: | The proposed transformation layer recovers 95–99% of the retrieval recall of a full re-embedding, adding less than 10,s query latency. |