Papers by Lingyuan Liu
Staged Knowledge Distillation Through Least-to-Most Prompting: Optimizing Teacher Guidance via Difficulty-Aware Training (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) enables the compression of large language models (LLMs) conventional methods suffer from training-inference mismatches and suboptimal performance due to expensive student-generated outputs. |
| Approach: | They propose a method that combines a CL strategy and adaptive loss design to reduce training mismatches and suboptimal performance. |
| Outcome: | L2M-KD outperforms existing white-box KD methods on instruction-following tasks . it outperformed existing methods, achieving superior student model performance with reduced overhead . |
Calibrated Progressive Distillation: Co-Designing Curriculum and Target Mixing for Knowledge Distillation of Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for knowledge distillation address the teacher–student capacity gap by mixing teacher and student distributions in the distillation target or using curriculum learning to sequence training from easy to hard examples. |
| Approach: | They propose a white-box KD framework that co-designs curriculum scheduling and target mixing through a unified difficulty-aware principle. |
| Outcome: | The proposed framework outperforms existing methods while reducing training runtime by over 10%. |
GOLFer: Smaller LMs-Generated Documents Hallucination Filter & Combiner for Query Expansion in Information Retrieval (2025.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs)-based query expansion for information retrieval necessitates larger, more advanced LLMs. |
| Approach: | They propose a method leveraging smaller open-source LMs for query expansion that augments queries with generated hypothetical documents with LLMs. |
| Outcome: | The proposed method outperforms existing methods on three web search and ten low-resource datasets and maintains competitive performance against larger LLMs. |
Exp4Fuse: A Rank Fusion Framework for Enhanced Sparse Retrieval using Large Language Model-based Query Expansion (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown potential in generating hypothetical documents for query expansion, but their efficacy is dependent on the quality of the generated documents. |
| Approach: | They propose a fusion ranking framework that uses zero-shot LLM-based query expansion to enhance sparse retrieval. |
| Outcome: | The proposed framework surpasses existing LLM-based query expansion methods on three MS MARCO-related datasets and seven low-resource datasets. |
Learning from Evolving Training Dynamics: An Entropy-Maximizing Data Curation Strategy for LLM Supervised Post-Training (2026.acl-long)
Copied to clipboard
| Challenge: | EVO-Curate is a dynamic data curation framework that synchronizes sample complexity with the maturing capacity of the Large Language Model (LLM). |
| Approach: | They propose a dynamic data curation framework that synchronizes sample complexity with maturing capacity of the Large Language Model (LLM) they use an Adaptive Dynamics Measurer to synthesize instantaneous difficulty and historical variability into a multidimensional utility score. |
| Outcome: | The proposed framework outperforms standard training baselines and traditional CL methods on instruction following, mathematical reasoning, and code generation architectures while maintaining manageable computational overhead. |