MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment (2026.acl-industry)
Copied to clipboard
Hanxian Huang, Igor Fedorov, Andrey Gromov, Bernard Beckerman, Naveen Suda, David Eriksson, Maximilian Balandat, Rylan Conway, Patrick Huber, Chinnadhurai Sankar, Ayushi Dalmia, Zechun Liu, Lemeng Wu, Tarek Elgamal, Adithya Sagar, Vikas Chandra, Raghuraman Krishnamoorthi
| Challenge: | MobileLLM-Flash is a family of foundation models for efficient on-device use with strong capabilities. |
| Approach: | They propose a method for designing on-device large language models under mobile latency constraints using hardware-in-the-loop architecture search. |
| Outcome: | The proposed model is amenable to industry-scale deployment and is compatible with mobile runtimes like Executorch. |
Similar Papers
Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM (2026.findings-acl)
Copied to clipboard
Sravanth Kodavanti, Sowmya Vajrala, Srinivas Soumitri Miriyala, Utsav Tiwari, Uttam Kumar, Utkarsh Kumar Mahawar, Achal Pratap Singh, Arya D, Narendra Mutyala, Vikram Nelvoy Rajendiran, Sharan Kumar Allur, Euntaik Lee, Dohyoung Kim, HyeonSu Lee, Gyusung Cho, JungBae Kim
| Challenge: | a framework for efficient on-device inference of large language models is needed for smartphones . memory, latency, and runtime flexibility are constraints for large language model deployments. |
| Approach: | They propose a hardware-aware framework for efficient on-device inference of a LLaMA-based multilingual foundation model for Samsung Galaxy S24 and S25 devices with SM8650 and SM8750 chipsets respectively. |
| Outcome: | The proposed framework improves memory, latency and performance across 9 languages and 8 tasks. |
LLM in a flash: Efficient Large Language Model Inference with Limited Memory (2024.acl-long)
Copied to clipboard
Keivan Alizadeh, Seyed Iman Mirzadeh, Dmitry Belenko, S. Khatamifard, Minsik Cho, Carlo C Del Mundo, Mohammad Rastegari, Mehrdad Farajtabar
| Challenge: | Large language models (LLMs) have high computational and memory requirements, especially for devices with limited memory. |
| Approach: | They propose a method that stores model parameters in flash memory but brings them on demand to DRAM . authors propose two techniques to optimize for reading data in larger, more contiguous chunks . |
| Outcome: | The proposed method reduces the volume of data transferred from flash and reads data in larger, more contiguous chunks. |
MobileQuant: Mobile-friendly Quantization for On-device Language Models (2024.findings-emnlp)
Copied to clipboard
Fuwen Tan, Royson Lee, Łukasz Dudziak, Shell Xu Hu, Sourav Bhattacharya, Timothy Hospedales, Georgios Tzimiropoulos, Brais Martinez
| Challenge: | Large language models (LLMs) have revolutionized language processing, but deployment on edge devices is costly in terms of memory, computation and energy. |
| Approach: | They propose to reduce the number of bits used to represent weights and activations . they propose to use 8-bit activations to enable LLMs to fully exploit mobile-friendly hardware . |
| Outcome: | The proposed method reduces the number of bits used to represent weights and activations . 8-bit activations are attractive for on-device deployment as they would exploit mobile-friendly hardware . |
EfficientLLM: Unified Pruning-Aware Pretraining for Auto-Designed Compact Language Models (2026.acl-long)
Copied to clipboard
Xingrun Xing, Zheng Liu, Shitao Xiao, Boyan Gao, Yiming Liang, Haokun Lin, Xianlin Zeng, Guoqi Li, Jiajun Zhang
| Challenge: | Large language models (LLMs) driven by scaling laws can be developed in large model sizes. |
| Approach: | They propose a pruning-aware pretraining approach that decouples LLM pruning from direct pretraining. |
| Outcome: | The proposed model outperforms pretraining models with 100M 1B parameters in commen sense benchmarks. |
MobiZO: Enabling Efficient LLM Fine-Tuning at the Edge via Inference Engines (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are currently pre-trained and fine-tuned on large cloud servers . fine-timing on resource-constrained edge devices presents significant memory and computational demands . |
| Approach: | They propose a resource-efficient fine-tuning framework for LLMs specifically designed for edge devices. |
| Outcome: | Experiments show that MobiZO achieves substantial runtime speedups and memory savings while improving fine-tuning accuracy. |
LinguaLinked: Distributed Large Language Model Inference on Mobile Devices (2024.acl-demos)
Copied to clipboard
| Challenge: | Recent research shows that large language models demonstrate enhanced capabilities in various language tasks. |
| Approach: | They introduce a system for decentralized, distributed LLM inference on mobile devices . they use optimized model assignment technique to segment LLMs and linear optimization to align segments with each device . |
| Outcome: | The proposed system performs well on high-end to low-end Android devices. |
Memory-Efficient Backpropagation for Fine-Tuning LLMs on Resource-Constrained Mobile Devices (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existing work on memory-efficient on-device fine-tuning of large language models with backpropagation has focused on approximating gradients with zeroth-order optimization (ZO). |
| Approach: | They propose a memory-efficient implementation of backpropagation on mobile devices that allows flexible trade-offs between memory usage and compute time while converging faster. |
| Outcome: | The proposed method can fine-tune LLMs with backpropagation using less than 1GB of memory while achieving better performance than the baseline. |
Scaling Down, Serving Fast: Compressing and Deploying Efficient LLMs for Recommendation Systems (2025.emnlp-industry)
Copied to clipboard
Kayhan Behdin, Ata Fatahibaarzi, Qingquan Song, Yun Dai, Aman Gupta, Zhipeng Wang, Hejian Sang, Shao Tang, Gregory Dexter, Sirou Zhu, Siyu Zhu, Tejas Dharamsi, Vignesh Kothapalli, Zhoutong Fu, Yihan Cao, Pin-Lun Hsu, Fedor Borisyuk, Natesh S. Pillai, Luke Simon, Rahul Mazumder
| Challenge: | Large language models (LLMs) have demonstrated remarkable performance across a wide range of industrial applications. |
| Approach: | They propose two techniques for training and deploying small language models that deliver high performance for a variety of industry use cases. |
| Outcome: | The proposed techniques retain much of the quality of larger models while reducing training/serving costs and latency. |
ModularMoE: Fast LLM Customization with Parameter-Sharing Mixture-of-Experts for Low-Resource Settings (2026.findings-acl)
Copied to clipboard
Jiaxing Liu, Qi Qi, Haifeng Sun, Dunjun Li, Zirui Zhuang, Bo He, Xiang Yang, Cong Liu, Jianxin Liao, Jingyu Wang
| Challenge: | Large Language Models impose significant computational and storage burdens on personal devices . existing customization approaches incur excessive computational costs or lead to suboptimal performance . |
| Approach: | They propose a training framework that converts pre-trained LLMs into parameter-sharing MoE models for lightweight deployment. |
| Outcome: | The proposed training framework outperforms state-of-the-art training frameworks at the same sparsity level while delivering up to 2.71 inference speedup. |
LLMs on a Budget? Say HOLA (2025.emnlp-industry)
Copied to clipboard
Zohaib Hasan Siddiqui, Jiechao Gao, Ebad Shabbir, Mohammad Anas Azeez, Rafiq Ali, Gautam Siddharth Kashyap, Usman Naseem
| Challenge: | Current solutions such as quantization, pruning, and Retrieval-Augmented Generation (RAG) offer only partial optimizations and often sacrifice accuracy, speed, or generality. |
| Approach: | They propose an end-to-end optimization framework for efficient LLM deployment . it leverages Hierarchical Speculative Decoding (HSD) for faster inference without quality loss. |
| Outcome: | HOLA delivers +17.6% EMA on GSM8K, +10.5% MCA on ARC, and reduced latency and memory on edge devices like Jetson Nano. |