Papers by Rodrigo Nogueira
MonoByte: A Pool of Monolingual Byte-level Language Models (2022.coling-1)
Copied to clipboard
| Challenge: | Existing studies have shown that multilingual models can achieve zero-shot cross-lingual performance on various NLP tasks, but due to the cost of pretraining, they often use public models with limited budgets. |
| Approach: | They propose to use tokenized models to test cross-lingual ability in multilingual and monolingual corpora. |
| Outcome: | The results show that models pretrained on multilingual and even monolingual corpora perform better than models pre-trained on SOTA models. |
Pretrained Transformers for Text Ranking: BERT and Beyond (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | This tutorial provides an overview of text ranking using neural network architectures known as transformers. |
| Approach: | This tutorial provides an overview of text ranking with neural network architectures known as transformers. |
| Outcome: | This tutorial provides an overview of text ranking with neural network architectures known as transformers. |
Designing Templates for Eliciting Commonsense Knowledge from Pretrained Sequence-to-Sequence Models (2020.coling-main)
Copied to clipboard
| Challenge: | Existing approaches to extract implicit knowledge from pretrained models are still unclear. |
| Approach: | They propose to use a template-based approach to extract implicit knowledge for commonsense reasoning on multiple-choice questions. |
| Outcome: | The proposed template can be extended to other MC tasks with contexts such as supporting facts in open-book question answering settings. |
Document Ranking with a Pretrained Sequence-to-Sequence Model (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results on the MS MARCO passage ranking task show that our ranking approach is superior to strong encoder-only models. |
| Approach: | They propose to use a pretrained sequence-to-sequence model to generate relevance labels as "target tokens" they also show how the underlying logits of these target tokens can be interpreted as relevance probabilities for ranking. |
| Outcome: | The proposed model outperforms existing models in a data-poor setting and significantly outperformed an encoder-only model on the MS MARCO passage ranking task. |