Papers by Chunsheng Zuo
GRPO-LEAD: A Difficulty-Aware Reinforcement Learning Approach for Concise Mathematical Reasoning in Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for group-relative policy optimization face challenges in reward sparsity, verbosity and inadequate focus on problem difficulty. |
| Approach: | They propose a method to improve group relative policy optimization with length-regularized rewards and explicit penalties for incorrect solutions. |
| Outcome: | The proposed method achieves state-of-the-art performance for 14B-scale models . it improves reasoning accuracy, conciseness, and efficiency . |
Position Information Emerges in Causal Transformers Without Positional Encodings via Similarity of Nearby Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Recent results suggest that positional encodings are not necessary when training decoder-only Transformer language models. |
| Approach: | They propose a causal attention mechanism that allows Transformers to store positional information without positional encodings. |
| Outcome: | The proposed model can reconstruct the positions of tokens without positional encodings. |