Papers by Vithursan Thangarasa
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models (2024.findings-acl)
Copied to clipboard
| Challenge: | Large language models are typically trained on general source data forvarious domains, but domain-specific pre-training is expensive and requires computational costs. |
| Approach: | They propose a suite of biomedicalLMs that leverage sparse pre-training on domain-specific biomedically text data. |
| Outcome: | The proposed model outperforms existing LLMs on biomedical tasks by 22.5x . |
MASSV: Multimodal Adaptation and Self-Data Distillation for Speculative Decoding of Vision-Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Speculative decoding of vision-language models provides a novel way to accelerate language model inference by enabling a lightweight draft model to propose multiple tokens that a larger target model verifies simultaneously. |
| Approach: | They propose a technique that allows a lightweight draft model to propose multiple tokens that a larger target model verifies simultaneously. |
| Outcome: | The proposed technique increases accepted length by 30% and delivers speedups of up to 1.46x compared to conventional text-only drafting baselines on visually-grounded tasks. |
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation (2026.acl-long)
Copied to clipboard
| Challenge: | Speculative decoding (SD) has proven to be effective for autoregressive generation in large language models (LLMs), however its application to vision-language models (VLMs) remains relatively unexplored. |
| Approach: | They propose a Speculative Decoding framework for vision-language models that integrates a neural architecture search framework and target-aware supernet training to identify optimal interaction strategies. |
| Outcome: | DREAM-S achieves 3.85 speedup compared to baselines on well-established vision-language models. |