Papers by Changdi Yang
Rethinking Token Reduction for State Space Models (2024.emnlp-main)
Copied to clipboard
Zheng Zhan, Yushu Wu, Zhenglun Kong, Changdi Yang, Yifan Gong, Xuan Shen, Xue Lin, Pu Zhao, Yanzhi Wang
| Challenge: | Existing methods for token reduction for SSMs lead to performance drops . a recent study shows that Mamba-2 improves the accuracy of the model by 5.7% to 13.1% . |
| Approach: | They propose a token reduction method that integrates token importance and similarity into SSMs and takes advantage of pruning and merging. |
| Outcome: | The proposed method improves accuracy by 5.7% to 13.1% on six benchmarks with Mamba-2 compared to existing methods while reducing computational demands and memory requirements. |