Papers by Machel Reid
VCDM: Leveraging Variational Bi-encoding and Deep Contextualized Word Representations for Improved Definition Modeling (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches for definition modeling combine distributional and lexical semantics in an implicit rather than direct way. |
| Approach: | They propose a model that introduces a continuous latent variable to model the relationship between a phrase and its definition. |
| Outcome: | The proposed model achieves state-of-the-art performance on four challenging benchmarks and the first non-English corpus. |
AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing reproducible benchmarks for machine translation are limited to high-resource or well-represented languages. |
| Approach: | They propose to use AfroMT to develop a reproducible machine translation benchmark for eight widely spoken African languages and a suite of analysis tools to take into account their unique properties. |
| Outcome: | The proposed benchmarks show significant improvements when pretraining on 11 languages, with gains of up to 2 BLEU points over strong baselines. |
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for multilingual sequence-to-sequence pretraining rely on monolingual corpora and do not use strong cross-lingual signal contained in parallel data. |
| Approach: | They propose a method that replaces monolingual words with a bilingual dictionary and predicts the reference translation according to a parallel corpus instead of recovering the original sequence. |
| Outcome: | The proposed method improves machine translation and cross-lingual natural language inference by 2.0 BLEU points and 6.7 accuracy points over existing methods at a fraction of their computational cost. |
BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer (2024.naacl-long)
Copied to clipboard
Akari Asai, Sneha Kudugunta, Xinyan Yu, Terra Blevins, Hila Gonen, Machel Reid, Yulia Tsvetkov, Sebastian Ruder, Hannaneh Hajishirzi
| Challenge: | Recent advances in few-shot generalization in natural language processing focus on English. |
| Approach: | They propose a benchmark that unifies 15 diverse tasks across 54 languages in a sequence-to-sequence format and provides a fixed set of few-shot examples and instructions. |
| Outcome: | The proposed framework unifies 15 diverse tasks across 54 languages in a sequence-to-sequence format and provides a fixed set of few-shot examples and instructions. |
Learning to Model Editing Processes (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing sequence generation models produce outputs in one pass, usually left-to-right . current models model only a single edit step, and do not fully model editing . |
| Approach: | They propose to model editing processes, modeling the whole process of iteratively generating sequences. |
| Outcome: | The proposed model improves performance on a variety of axes compared to previous models . iterative refinement and editing are central parts of human creative workflow . |
On the Role of Parallel Data in Cross-lingual Transfer Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing multilingual models do not exploit the full potential of monolingual data, a new study finds . prior work has shown that parallel data is beneficial for cross-lingual learning, but it is unclear if it is the data itself or the modeling of parallel interactions that matters. |
| Approach: | They compare unsupervised machine translation to supervised machine translator and gold parallel data to generate synthetic parallel data. |
| Outcome: | The proposed model generated parallel data is better than supervised machine translation and gold parallel data in both general and task-specific settings. |
mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent large language models display surprising multilingual capabilities despite being pre-trained on English data. |
| Approach: | They propose a multilingual sequence-to-sequence model that disentangles language-specific information from language-agnostic information. |
| Outcome: | The proposed model outperforms existing models on representative natural language understanding and generation tasks in 40+ languages. |
LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent work on text style transfer proposes single-span editing as an alternative to generating the target text from scratch. |
| Approach: | They propose a coarse-to-fine editor for style transfer that transforms text using Levenshtein edit operations (e.g. insert, replace, delete). |
| Outcome: | The proposed method outperforms existing methods on sentiment and politeness transfer and improves model performance. |
Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent improvements in NLP tasks can be attributed to the Transformer model. |
| Approach: | They propose to use parameter-sharing methods to reduce parameter budgets in generative models by using sandwich-style parameter sharing and self-attentive embedding factorization. |
| Outcome: | The proposed model outperforms the current RNN model even with significantly fewer parameters. |
A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translation (2022.naacl-main)
Copied to clipboard
David Adelani, Jesujoba Alabi, Angela Fan, Julia Kreutzer, Xiaoyu Shen, Machel Reid, Dana Ruiter, Dietrich Klakow, Peter Nabende, Ernie Chang, Tajuddeen Gwadabe, Freshia Sackey, Bonaventure F. P. Dossou, Chris Emezue, Colin Leong, Michael Beukman, Shamsuddeen Muhammad, Guyo Jarso, Oreen Yousuf, Andre Niyongabo Rubungo, Gilles Hacheme, Eric Peter Wairagala, Muhammad Umair Nasir, Benjamin Ajibade, Tunde Ajayi, Yvonne Gitau, Jade Abbott, Mohamed Ahmed, Millicent Ochieng, Anuoluwapo Aremu, Perez Ogayo, Jonathan Mukiibi, Fatoumata Ouoba Kabore, Godson Kalipe, Derguene Mbaye, Allahsera Auguste Tapo, Victoire Memdjokam Koagne, Edwin Munkoh-Buabeng, Valencia Wagner, Idris Abdulmumin, Ayodele Awokoya, Happy Buzaaba, Blessing Sibanda, Andiswa Bukula, Sam Manthalu
| Challenge: | Low-resource languages are left out of large-scale pretraining datasets . authors explore how to leverage existing pre-trained models to create low-resourced translation systems for 16 African languages. |
| Approach: | They investigate how large-scale pre-trained models can be used to create low-resource translation systems for 16 African languages. |
| Outcome: | The proposed models can translate between hundreds of languages even though there is little parallel data available for training. |
M2D2: A Massively Multi-Domain Language Modeling Dataset (2022.emnlp-main)
Copied to clipboard
| Challenge: | M2D2 consists of 8.5B tokens and spans 145 domains extracted from Wikipedia and Semantic Scholar. |
| Approach: | They propose to organize 145 domains into 22 groups and use ontologies from Wikipedia and ArXiv to study domain adaptation in language models. |
| Outcome: | The proposed model enables the study of domain adaptation in language models (LMs) it shows that small amounts of fine-grained data can lead to larger in-domain performance gains than weakly relevant data. |