Papers by Hanxian Huang
Learning to Maximize Mutual Information for Chain-of-Thought Distillation (2024.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation is a technique of transferring knowledge from large, complex models to smaller ones. |
| Approach: | They propose a method utilizing chain-of-thought distillation to transfer knowledge from large, complex models to smaller ones by maximizing mutual information of the representation features of the two tasks. |
| Outcome: | The proposed method outperforms the state-of-the-art knowledge distillation method on four datasets. |
You Only Use Reactive Attention Slice When Retrieving From Long Context (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing retrieval techniques for language models are limited due to their reliance on lexical similarity and are computationally expensive to train. |
| Approach: | They propose a training-free and fine-tuning-free attention-based retrieval technique that uses a reaction score heuristic to quantify how an LM’s self-attention “reacts” to a user query. |
| Outcome: | The proposed approach improves QA task accuracy by 15% and inference throughput by 31% compared to embedding-based retrieval. |
MobileLLM-Flash: Latency-Guided On-Device LLM Design for Industry Scale Deployment (2026.acl-industry)
Copied to clipboard
Hanxian Huang, Igor Fedorov, Andrey Gromov, Bernard Beckerman, Naveen Suda, David Eriksson, Maximilian Balandat, Rylan Conway, Patrick Huber, Chinnadhurai Sankar, Ayushi Dalmia, Zechun Liu, Lemeng Wu, Tarek Elgamal, Adithya Sagar, Vikas Chandra, Raghuraman Krishnamoorthi
| Challenge: | MobileLLM-Flash is a family of foundation models for efficient on-device use with strong capabilities. |
| Approach: | They propose a method for designing on-device large language models under mobile latency constraints using hardware-in-the-loop architecture search. |
| Outcome: | The proposed model is amenable to industry-scale deployment and is compatible with mobile runtimes like Executorch. |