Papers by Marie-Anne Lachaux
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data (2020.lrec-1)
Copied to clipboard
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, Vishrav Chaudhary, Francisco Guzmán, Armand Joulin, Edouard Grave
| Challenge: | Pre-training text representations have led to significant improvements in many areas of natural language processing. |
| Approach: | They propose a pipeline to extract monolingual datasets from Common Crawl . pipeline follows data processing introduced in fastText that deduplicates documents . |
| Outcome: | The proposed pipeline performs standard document deduplication and language identification similar to the pipeline introduced in fastText and a filtering step to select documents close to high quality corpora like Wikipedia. |
Target Conditioning for One-to-Many Generation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Neural Machine Translation models lack diversity in their generated translations, even when paired with search algorithm, like beam search. |
| Approach: | They propose to model one-to-many mapping by conditioning a decoder on a latent variable that represents the domain of target sentences. |
| Outcome: | The proposed method can scale to any number of domains without affecting performance or training time. |