Papers with IIR
Focus Your Attention (with Adaptive IIR Filters) (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to deal with long-range data processing are implicit convolutions and regularized parameterization. |
| Approach: | They propose a new layer where dynamic (i.e., input-dependent) IIR filters are used to process the input sequence prior to applying conventional attention. |
| Outcome: | The proposed layer performs on-par with state-of-the-art networks with a fraction of their parameters and time complexity that is sub-quadratic with input size. |