Papers by Soumyadeep Roy
Learning Faster with Better Tokens: Parameter-Efficient Vocabulary Adaptation for Specialized Text Summarization (2026.acl-long)
Copied to clipboard
| Challenge: | a new approach to adapt generalist models to expert domains is needed to overcome this problem. |
| Approach: | They propose a parameter-efficient domain adaptation approach that combines vocabulary adaptation with pretraining for LLM-based text summarization. |
| Outcome: | The proposed approach reduces training time by 35-55% over continual pretraining and reduces parameter counts up to 37% w.r.t expansion-only methods. |
Adaptive BPE Tokenization for Enhanced Vocabulary Adaptation in Finetuning Pretrained Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Current vocabulary adaptation approaches append the target domainspecific vocabulary (V DOMAIN) at the end of the PLM vocabulary. |
| Approach: | They propose a vocabulary adaptation scheme that appends a target domain-specific vocabulary (V DOMAIN) at the end of the PLM vocabulary. |
| Outcome: | The proposed approach improves by 3.57% (in terms of accuracy) and 1.87% (royal-L) over various classification and summarization tasks. |
Evaluation of LLMs in Medical Text Summarization: The Role of Vocabulary Adaptation in High OOV Settings (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been successful in medical text summarization . however, they do not perform fine-grained evaluations under difficult settings . |
| Approach: | They show that large language models show a significant performance drop for data points with high concentration of out-of-vocabulary words or with high novelty. |
| Outcome: | The proposed model shows a significant performance drop for data points with high concentration of out-of-vocabulary words or with high novelty. |