Papers by Heshan Liu
Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a large computational cost for attention computation in large language models is a major obstacle . |
| Approach: | They propose a convolution-like structure for attention computation using convolution matrices . they then propose an efficient approximation method to approximate the attention matrix . |
| Outcome: | The proposed method achieves nearly linear time complexity in n1+o(1) time. |