Papers by Huazhe Xu
TemplateRL: Structured Template-Guided Reinforcement Learning for LLM Reasoning (2026.findings-acl)
Copied to clipboard
Jinyang Wu, Chonghua Liao, Mingkuan Feng, Shuai Zhang, Zhengqi Wen, Haoran Luo, Ling Yang, Huazhe Xu, Jianhua Tao
| Challenge: | Existing RL methods rely on unstructured self-sampling to fit scalar rewards, resulting in inefficient rollouts. |
| Approach: | They propose a structured template-guided RL framework that augments policy optimization with explicit template guidance. |
| Outcome: | Experiments show that TemplateRL outperforms GRPO and GRPI by 99% on AIME and 41% on AMC with superior stability on weak models and remarkable cross-domain generalization. |
World Models with Hints of Large Language Models for Goal Achieving (2025.naacl-long)
Copied to clipboard
| Challenge: | Existing methods address this by adding intrinsic rewards, but they fail to provide meaningful guidance in long-horizon decision-making tasks with large state and action spaces lacking purposeful exploration. |
| Approach: | They propose a multi-modal model-based RL approach that integrates the proposed hinting subgoals into the model rollouts to encourage goal discovery and reaching in challenging tasks. |
| Outcome: | The proposed model outperforms existing methods in challenging, sparse-reward environments such as HomeGrid, Crafter, and Minecraft by 41.8%, 21.1%, and 9.9%. |