Papers by Zhonghou Lv
Token-Level Policy Optimization: Linking Group-Level Rewards to Token-Level Aggregation via sequence-level likelihood (2026.acl-long)
Copied to clipboard
| Challenge: | Group Relative Policy Optimization (GRPO) has significantly advanced the reasoning ability of large language models (LLMs). |
| Approach: | They propose a token-level framework that leverages sequence-level likelihood to link group-level rewards with individual tokens via token- level aggregation and introduces a KL-Divergence mask constraint that targets tokens with positive advantages and decreasing entropy to mitigate abrupt policy updates. |
| Outcome: | Experiments show that TEPO achieves state-of-the-art performance on mathematical reasoning benchmarks and reduces convergence time by 50% compared with GRPO/DAPO. |
Safety-Utility Conflicts Are Not Global: Surgical Alignment via Head-Level Diagnosis (2026.acl-long)
Copied to clipboard
| Challenge: | Existing mitigation strategies rely on global gradient geometry to resolve alignment conflicts . however, they overlook Modular Heterogeneity within Transformers, resulting in suboptimal trade-offs . Conflict-Aware Sparse Tuning (CAST) combines head-level diagnosis with sparse fine-tuning . |
| Approach: | They propose a framework that integrates head-level diagnosis with sparse fine-tuning to address this limitation. |
| Outcome: | The proposed framework integrates head-level diagnosis with sparse fine-tuning to reduce alignment conflicts in LLMs. |