Papers by Runsong Zhao
Read As Human: Compressing Context via Parallelizable Close Reading and Skimming (2026.acl-long)
Copied to clipboard
Jiwei Tang, Shilei Liu, Zhicheng Zhang, Qingsong Lv, Runsong Zhao, Tingwei Lu, Langming Liu, Haibin Chen, Yujin Yuan, Hai-Tao Zheng, Wenbo Su, Bo Zheng
| Challenge: | Existing task-aware methods require loading the entire input sequence at once for compression, which suffer from computational inefficiency. |
| Approach: | They propose a framework that adopts an adaptive hybrid reading strategy to reduce computational inefficiency and redundant information in long-context scenarios. |
| Outcome: | Experiments show that RAM outperforms baselines on multiple question answering and summarization benchmarks while delivering up to a 12x speedup on long inputs. |
Position IDs Matter: An Enhanced Position Layout for Efficient Context Compression in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to compress context information ignore holistic contextual dependencies. |
| Approach: | They propose a method that adjusts position encodings to minimize the distance between context tokens and special tokens. |
| Outcome: | Enhanced Position Layout (EPL) improves compression of context information in large language models. |
Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models (2024.emnlp-main)
Copied to clipboard
Xinyu Liu, Runsong Zhao, Pengcheng Huang, Chunyang Xiao, Bei Li, Jingang Wang, Tong Xiao, JingBo Zhu
| Challenge: | Existing methods, tasks and benchmarks to measure model’s effective memory length are limited. |
| Approach: | They propose a method called forgetting curve to measure the memorization capability of long-context models. |
| Outcome: | The proposed method is robust to the tested corpus and experimental settings, and can be applied to any model size. |
CoMeT: Collaborative Memory Transformer for Efficient Long Context Modeling (2026.acl-long)
Copied to clipboard
Runsong Zhao, Shilei Liu, Jiwei Tang, Langming Liu, Haibin Chen, Weidong Zhang, Yujin Yuan, Tong Xiao, JingBo Zhu, Wenbo Su, Bo Zheng
| Challenge: | a novel architecture that enables LLMs to handle arbitrarily long sequences with constant memory usage and linear time complexity is a major barrier to long-context processing. |
| Approach: | They propose a novel architecture that enables LLMs to handle arbitrarily long sequences with constant memory usage and linear time complexity. |
| Outcome: | The proposed architecture can handle arbitrarily long sequences with constant memory usage and linear time complexity. |