Papers by Jae-young Jo
Roles and Utilization of Attention Heads in Transformer-based Neural Language Models (2020.acl-main)
Copied to clipboard
| Challenge: | Sentence encoders based on transformer architectures have shown promising results on various natural language understanding tasks. |
| Approach: | They propose a sentence representation method that takes advantage of most influential attention heads. |
| Outcome: | The proposed method improves performance on the downstream tasks. |