Papers by Qijun Miao
User-side Model Consistency Monitoring for Open Source Large Language Models Inference Services (2025.acl-long)
Copied to clipboard
| Challenge: | Open-source large language models have attracted substantial user base by offering quality comparable to closed-source models at a significantly lower cost. |
| Approach: | They propose a paradigm for model consistency monitoring on the user side that constructs metrics based on logits produced by LLMs to differentiate sequences generated by degraded models. |
| Outcome: | The proposed method is implementable on consumer-grade devices and demonstrates that it is cost-effective and reliable. |