Papers by Jungmin Yun
Focus on the Core: Efficient Attention via Pruned Token Compression for Document Classification (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained transformers suffer from a computationally expensive self-attention mechanism that interacts with all tokens, including those unfavorable to classification performance. |
| Approach: | They propose to integrate token pruning and token combining strategies to improve model performance and reduce computational demands. |
| Outcome: | Experiments with various datasets show that the proposed model performs better than baseline models, with the best improvement over the existing model. |