Papers by Jusen Du
Native Hybrid Attention for Efficient Sequence Modeling (2026.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that NHA surpasses Transformers on recall-intensive tasks. |
| Approach: | They propose a hybrid architecture of linear and full attention that integrates both into a unified layer design. |
| Outcome: | The proposed architecture surpasses Transformers and other hybrid baselines on recall-intensive and commonsense reasoning tasks. |