Papers by Ren Pang
Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) increase test-time computation, often in the form of chain-of-thought (CoT) however, reasoning traces can become unnecessarily long, increasing computation costs without improving accuracy and sometimes even degrading performance. |
| Approach: | They propose a multi-stage efficient reasoning method that combines supervised fine-tuning with reinforcement learning using an adaptive length penalty. |
| Outcome: | The proposed method reduces response length by an average of 28% for 8B models and 40% for 32B models while incurring only minor performance drops of 1.6 and 2.5 points, respectively. |
Model Balancing Helps Low-data Training and Fine-tuning (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent advances in foundation models have emphasized the need to align pre-trained models with specialized domains using small, curated datasets. |
| Approach: | They propose a layer-wise learning rate scheduler that balances training quality across layers . they adapt it to a curated dataset to achieve alignment with specialized domains . |
| Outcome: | The proposed model shows that it can be used to balance training quality across layers and improve low-data training and fine-tuning for both NLP and SciML tasks. |