Papers by Ashish Sirasao
Recover-LoRA: Data-Free Accuracy Recovery of Degraded Language Models via Low-Rank Adaptation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Inference optimizations such as quantization, pruning, format and datatype conversion, model export, and serialization can lead to functional degradations in language model task performance. |
| Approach: | They propose a lightweight and dataset-agnostic method to recover model accuracies from quantization, pruning, format and datatype conversion, model export, and serialization errors. |
| Outcome: | The proposed method recovers model accuracies by 5-17% on MHA and GQA models. |
KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference (2026.eacl-industry)
Copied to clipboard
| Challenge: | Long-context Large Language Models (LLMs) face significant memory bottlenecks due to the linear growth of key-value (KV) cache with sequence length. |
| Approach: | They propose a framework that maps the trade-off frontier between total memory consumption and task accuracy across three complementary optimization techniques. |
| Outcome: | The proposed model-specific configurations achieve 68-78% total memory reduction with minimal (1-3%) accuracy degradation on long-context tasks. |
Enhancing One-Shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanism (2025.coling-main)
Copied to clipboard
Guanchen Li, Xiandong Zhao, Lian Liu, Zeping Li, Yixing Xu, Dong Li, Lu Tian, Jie He, Ashish Sirasao, Emad Barsoum
| Challenge: | Pre-trained language models (PLMs) are robust in contextual understanding but their considerable size incurs significant computational and storage costs. |
| Approach: | They propose a Sparse-Dense-Sparse pruning framework to prune PLMs . they prune less critical connections using conventional pruning methods . |
| Outcome: | The proposed pruning framework outperforms SparseGPT and Wanda under identical sparsity. |