Papers by Renke Shan
Re3Syn: A Dependency-Based Data Synthesis Framework for Long-Context Post-training (2025.acl-long)
Copied to clipboard
| Challenge: | Existing methods for constructing long-context data by concatenating short documents have overlooked a crucial characteristic of long-constituency data quality, semantic dependency. |
| Approach: | They propose a framework called Retrieval, Dependency Recognition, and Reorder for data synthesis which leverages semantic similarity to retrieve relevant documents and form several batches. |
| Outcome: | The proposed framework leverages semantic similarity to retrieve relevant documents and form several batches. |
CLaSp: In-Context Layer Skip for Self-Speculative Decoding (2025.acl-long)
Copied to clipboard
Longze Chen, Renke Shan, Huiming Wang, Lu Wang, Ziqiang Liu, Run Luo, Jiawei Wang, Hamid Alinejad-Rokny, Min Yang
| Challenge: | Existing methods for drafting Large Language Models require additional modules to be trained, which can be challenging to implement and ensure compatibility across various LLMs. |
| Approach: | They propose an in-context layer-skipping strategy for self-speculative decoding that uses a plug-and-play mechanism to skip intermediate layers of the verify model to construct a compressed draft model. |
| Outcome: | The proposed method achieves a speedup of 1.3 1.7 on LLaMA3 series models without altering the original distribution of the generated text. |