Papers by Lujun Li
How LLMs React to Industrial Spatio-Temporal Data? Assessing Hallucination with a Novel Traffic Incident Benchmark Dataset (2025.naacl-industry)
Copied to clipboard
Qiang Li, Mingkun Tan, Xun Zhao, Dan Zhang, Daoan Zhang, Shengzhao Lei, Anderson S. Chu, Lujun Li, Porawit Kamnoedboon
| Challenge: | Large language models (LLMs) have a high potential to digitize and enhance the health & public services industry. |
| Approach: | They propose to use a cross-lingual benchmark dataset to assess the robustness of state-of-the-art LLMs in the spatio vs temporal domain for traffic incident classification. |
| Outcome: | The proposed model performs well in the spatio-temporal domain and in the non-English context. |
BTC-LLM: Efficient Sub-1-Bit LLM Quantization via Learnable Transformation and Binary Codebook (2026.acl-long)
Copied to clipboard
Hao Gu, Lujun Li, Hao Wang, Lei Wang, Zheyu Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Sirui Han, Yike Guo
| Challenge: | Recent sparsity-aware binarization approaches can achieve sub-1-bit compression, but they face performance degradation, mask-management overhead, and limited hardware compatibility. |
| Approach: | They propose a binary quantization framework that leverages binary pattern clustering and weight transformation to overcome performance degradation and mask-management overhead. |
| Outcome: | The proposed framework achieves state-of-the-art compression (1.11–0.7 bits) it maintains high performance with only a 3.1% accuracy drop in zero-shot benchmarks while delivering a 1.6 speedup over FP16. |
Bit-by-Bit: Progressive QAT Strategy with Outlier Channel Splitting for Stable Low-Bit LLMs (2026.acl-long)
Copied to clipboard
Binxing Xu, Hao Gu, Lujun Li, Hao Wang, Bei Liu, Jiacheng Liu, Qiyuan Zhu, Xintong Yang, Chao Li, Sirui Han, Yike Guo
| Challenge: | Existing approaches to training LLMs at ultra-low precisions suffer from convergence instability and substantial training costs. |
| Approach: | They propose a progressive QAT framework with outlier channel splitting to address these issues . they use nested structure of integer quantization grids to enable a "train once, deploy any precision" paradigm . |
| Outcome: | The proposed framework outperforms baselines on both Llama2/3 and W2A16, with an 11 speedup over BF16. |
LPZero: Language Model Zero-cost Proxy Search from Zero (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing zero-cost (ZC) proxies rely on expert knowledge and incur significant trial-and-error costs. |
| Approach: | They propose a framework that automatically designs zero-cost (ZC) proxies for various tasks and incorporates genetic programming to find the optimal symbolic composition. |
| Outcome: | The proposed framework achieves higher ranking consistency than human-designed proxies on NLP tasks. |
Small Language Models in the Real World: Insights from Industrial Text Classification (2025.acl-industry)
Copied to clipboard
| Challenge: | With the emergence of ChatGPT, transformer-only models have significantly advanced text classification and related tasks. |
| Approach: | They propose to use prompt engineering and supervised fine-tuning methods for transformer-based text classification in industrial applications. |
| Outcome: | The proposed models perform well in a variety of industrial scenarios, including email classification, legal document categorization, and the classification of extremely long academic texts. |
QaRL: Rollout-Aligned Quantization-Aware RL for Fast and Stable Training under Training–Inference Mismatch (2026.findings-acl)
Copied to clipboard
Hao Gu, Hao Wang, Jiacheng Liu, Lujun Li, Qiyuan Zhu, Bei Liu, Binxing Xu, Lei Wang, Xintong Yang, Sida Lin, Sirui Han, Yike Guo
| Challenge: | Recent work has shown that reinforcement learning with simple rule-based reward functions (RLVR) can induce emergent reasoning behaviors and yield gains in challenging domains such as math problem solving. |
| Approach: | They propose a rollout-alignment-quantization-aware RL which aligns training-side forward with the quantized rollout to minimize mismatch. |
| Outcome: | The proposed approach outperforms quantized-rollout training by +5.5 on Qwen3-30B-A3B MoE for math problems while maintaining low-bit throughput. |
BayesKD: Bayesian Knowledge Distillation for Compact LLMs in Constrained Fine-tuning Scenarios (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have revolutionized various domains with their remarkable capabilities, but their massive parameter sizes pose significant challenges for fine-tuning and inference. |
| Approach: | They propose a Bayesian Knowledge Distillation framework for compact Large Language Models in resource-constrained fine-tuning scenarios that employs Logits Dual-Scaling, Knowledge Alignment Module, and Bayes Distillations Optimization. |
| Outcome: | The proposed framework outperforms baseline methods on various state-of-the-art LLMs, including LLaMA, Qwen2, Bloom, and Vicuna. |