Papers by Zhanchao Zhou
Value Residual Learning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing decoder-only transformers fail to preserve initial token-level information in deeper layers. |
| Approach: | They propose a new architecture that incorporates value residual connections in addition to hidden state residuals. |
| Outcome: | The proposed architecture reduces KV cache size by nearly half with only a small performance penalty and can be integrated with other KV-efficient methods. |
Every Token Counts: Generalizing 16M Ultra-Long Context in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | a recent study explores efficient ultra-long context modeling. |
| Approach: | They propose to use Hierarchical Sparse Attention to achieve efficient ultra-long context modeling. |
| Outcome: | The proposed model performs comparable to full-attention baselines on in-domain and out-of-domain tasks. |
Dynamics of Instruction Fine-Tuning for Chinese Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Instruction tuning is a burgeoning method to elicit the general intelligence of Large Language Models. |
| Approach: | They investigate the effects of data quantity, model size, and data construction methods on instruction tuning for Chinese LLMs. |
| Outcome: | The proposed model includes over 40,000 high-quality instruction instances covering ten underlying abilities. |