Papers by Daohai Yu
Training Long-Context LLMs Efficiently via Chunk-wise Optimization (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in long-context large language models have demonstrated superior retrieval quality compared to retrievalaugmented generation (RAG) approaches. |
| Approach: | They propose a memory-efficient training paradigm that partitions lengthy inputs into manageable chunks. |
| Outcome: | The proposed model expands maximum sequence length from 1K to 16K tokens on a single RTX 3090 GPU, while SpaCO achieves accelerated training speed. |