Papers by Kaiyang Wan
Flipping Knowledge Distillation: Leveraging Small Models’ Expertise to Enhance LLMs in Text Matching (2025.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated remarkable capabilities in acquiring diverse knowledge, making them highly effective across a wide range of tasks. |
| Approach: | They propose a flipped knowledge distillation paradigm where LLM learns from SLM . they propose to reinterpret LLMs as encoder-decoder models using LoRA . |
| Outcome: | The proposed model has been deployed in an online application environment and validated on financial and healthcare benchmarks and real-world applications. |
A Cognitive Writing Perspective for Constrained Long-Form Text Generation (2025.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models struggle to generate high-quality long-form text in a single pass . a new framework that trains LLMs to write human-like writing capabilities is needed . |
| Approach: | They propose a framework that equips large language models with human-like cognitive writing capabilities . they use a planning agent and multiple Generation Agents to generate long-form text in parallel . |
| Outcome: | CogWriter surpasses GPT-4o by 22% in complex instruction completion accuracy . the framework can generate coherent text in a single pass with fluency that rivals human writers . |
HAHE: Hierarchical Attention for Hyper-Relational Knowledge Graphs in Global and Local Level (2023.acl-long)
Copied to clipboard
Haoran Luo, Haihong E, Yuhao Yang, Yikai Guo, Mingzhi Sun, Tianyu Yao, Zichen Tang, Kaiyang Wan, Meina Song, Wei Lin
| Challenge: | Existing research on HKGs rarely models the graphical and sequential structure of HKG, limiting their representation. |
| Approach: | They propose a Hierarchical Attention model for HKG Embedding that includes global-level and local-level attention to model the graphical structure of HKGs. |
| Outcome: | The proposed model achieves state-of-the-art performance on HKG standard datasets and addresses the issue of HKG multi-position prediction for the first time. |