Papers by Bodun Hu
FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping (2024.emnlp-main)
Copied to clipboard
| Challenge: | Autoregressive Large Language Models (LLMs) are omnipresent but typically come with a substantial model size. |
| Approach: | They propose a novel fine-grained skip strategy for autoregressive large language models . they observe the saturation of computationally expensive feed-forward blocks of LLMs . |
| Outcome: | The proposed method can skip 25-30% of FFN blocks with marginal change in performance on knowledge-intensive generation tasks. |
MOSEL: Inference Serving Using Dynamic Modality Selection (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in machine learning have enabled deep learning to exceed human capabilities in various tasks. |
| Approach: | They propose a new form of dynamism, modality selection, where modality picks modalities from inference inputs while maintaining the model quality. |
| Outcome: | The proposed system improves system throughput by 3.6 and job completion times by 11 compared to modality-agnostic approaches. |
StitchLLM: Serving LLMs, One Block at a Time (2025.acl-long)
Copied to clipboard
Bodun Hu, Shuozhe Li, Saurabh Agarwal, Myungjin Lee, Akshay Jajoo, Jiamin Li, Le Xu, Geon-Woo Kim, Donghyun Kim, Hong Xu, Amy Zhang, Aditya Akella
| Challenge: | Existing techniques like distillation and pruning are not efficient for large language models. |
| Approach: | They propose a dynamic model routing framework that uses a powerful bottom model to process all queries and a lightweight routing mechanism to allocate computational resources appropriately. |
| Outcome: | The proposed framework improves system throughput while minimizing performance degradation. |