Papers by Julien Perez
Globalizing BERT-based Transformer Architectures for Long Document Summarization (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to fine-tune a large language model on downstream tasks show several limitations when the target task requires to reason with long documents. |
| Approach: | They propose a hierarchical approach where the input is divided in multiple blocks independently processed by the scaled dot-attentions and combined between the successive layers. |
| Outcome: | The proposed approach performs well on three extractive summarization corpora of scientific papers and news articles. |