Papers by Qibin Wang
DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding (2026.findings-acl)
Copied to clipboard
| Challenge: | Autoregressive (AR) decoding in large language models is latency-bounded by strictly sequential token generation. |
| Approach: | They propose a diffusion-based drafter that proposes multi-token candidates and then verifies them in parallel by the target model. |
| Outcome: | The proposed drafter generates multi-token proposals in a single forward pass while remaining compatible with standard AR verifiers. |
Anchoring-Guidance Fine-Tuning (AnGFT): Elevating Professional Response Quality in Role-Playing Conversational Agents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have demonstrated significant advancements in various fields, notably in Role-Playing Conversational Agents (RPCAs). |
| Approach: | They propose an Anchoring-Guidance Fine-Tuning Framework to integrate relevant expert knowledge into RPCAs' training process to mitigate this issue. |
| Outcome: | The proposed framework significantly improves the RPCAs’ performance in handling role-specific professional queries while preserving their robust role-playing abilities. |
Learning to Refine: Self-Refinement of Parallel Reasoning in LLMs (2026.findings-acl)
Copied to clipboard
Qibin Wang, Pu Zhao, Shaohan Huang, Fangkai Yang, Lu Wang, Furu Wei, Qingwei Lin, Saravan Rajmohan, Dongmei Zhang
| Challenge: | Existing approaches to test-time scaling are limited due to the quality of candidate responses. |
| Approach: | They propose a new metric to quantify the relative improvement of self-refinement beyond majority voting. |
| Outcome: | The proposed method achieves state-of-the-art performance across five benchmarks over other methods. |
Reinforcement Learning on Pre-Training Data (2026.acl-long)
Copied to clipboard
Siheng Li, Kejiao Li, Zenan Xu, Guanhua Huang, Kun Li, Haoyuan Wu, null Wujiajia, Zihao Zheng, Chenchen Zhang, Kun Shi, Xue Gong, Qi Yi, Ruibin Xiong, Tingqiang Xu, Yuhao Jiang, Jianfeng Yan, Yuyuan Zeng, Guanghui Xu, Jinbao Xue, Zhijiang xu, Zheng Fang, Shuai LI, Qibin Liu, Xiaoxue Li, Zhuoyu Li, Yangyu Tao, Fei Gao, Cheng Jiang, Bochao Wang, Kai Liu, Jianchen Zhu, Wai Lam, Bo Zhou, Di Wang
| Challenge: | Recent progress in large language models is driven by scaling of training compute through pre-training with nexttoken prediction (NTP) or post-training (RL) Pre-training using NTP enables models to acquire extensive knowledge and skills from general data, but it suffers from data inefficiency and catastrophic forgetting in continual learning settings. |
| Approach: | They propose to scale training compute through pre-training with next-token prediction (NTP) or post-training by scaling reinforcement learning (RL) to improve learning from general data. |
| Outcome: | Experiments on multiple benchmarks and models show that the proposed approach improves continual pre-training and provides a strong foundation for post-training on Qwen3-8B-Base. |
MR. Judge: Multimodal Reasoner as a Judge (2025.emnlp-main)
Copied to clipboard
| Challenge: | Effective reward modeling is especially valuable in reinforcement learning (RLHF) . |
| Approach: | They propose a paradigm for empowering general-purpose MLLMs judges with strong reasoning capabilities by using multiple-choice problem models instead of directly assigning scores. |
| Outcome: | The proposed model surpasses GPT-4o on VL-RewardBench and improves performance on MM-Vet by up to 7.7%. |
PMSS: Pretrained Matrices Skeleton Selection for LLM Fine-tuning (2025.coling-main)
Copied to clipboard
| Challenge: | Low-rank adaptation and its variants have been popular due to their ability to avoid excessive inference costs. |
| Approach: | They propose a low-rank adaptation method that enables high-rank updates with low costs while leveraging semantic and linguistic information inherent in pre-trained weight. |
| Outcome: | The proposed method outperforms LoRA and other fine-tuning methods across tasks with less trainable parameters. |