Papers by Wenrui Cai
Thinking with DistilQwen: A Tale of Four Distilled Reasoning and Reward Model Series (2025.emnlp-industry)
Copied to clipboard
| Challenge: | In the rapidly evolving landscape of large language models, the need for efficient reasoning models has become increasingly urgent. |
| Approach: | They extend the Qwen model family by introducing four model series specifically designed for industrial applications. |
| Outcome: | The proposed models outperform previous models in multiple benchmarks and provide scalable training and inference functionality on the Alibaba Cloud PAI platform. |
Enhancing Reasoning Abilities of Small LLMs with Cognitive Alignment (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to distill chain-of-thought (CoT) results from large language reasoning models (LRMs) to small models are ineffective and require substantial amount of annotated data. |
| Approach: | They propose a Critique-Rethink-Verify system for training small language reasoning models that can be critiquized according to the cognitive capabilities of smaller models. |
| Outcome: | The proposed system outperforms other methods on challenging reasoning benchmarks. |
EasyDistill: A Comprehensive Toolkit for Effective Knowledge Distillation of Large Language Models (2025.emnlp-demos)
Copied to clipboard
| Challenge: | In this paper, we present EasyDistill, a comprehensive toolkit designed for effective black-box and white-box knowledge distillation (KD) of large language models. |
| Approach: | They propose a toolkit for effective black-box and white-box knowledge distillation (KD) of large language models (LLMs). |
| Outcome: | The framework offers data synthesis, supervised fine-tuning, ranking optimization, and reinforcement learning techniques specifically tailored for KD scenarios. |
Reasoning with OmniThought: A Large CoT Dataset with Verbosity and Cognitive Difficulty Annotations (2026.acl-long)
Copied to clipboard
| Challenge: | Existing resources often fail to provide extensive reasoning problems with coherent CoT processes distilled from multiple teacher models. |
| Approach: | They propose a large-scale dataset featuring 2 million CoT processes generated by multiple powerful LRMs. |
| Outcome: | The proposed dataset features 2 million CoT processes and is validated by multiple powerful LRMs. |