Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Gomis, Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Elsa Sarrías, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, Jaume Zaragoza
| Challenge: | We describe methods to create the largest publicly available parallel corpora by crawling the web . parallel corpus is essential for building highquality machine translation systems . |
| Approach: | They describe methods to create largest publicly available parallel corpora by crawling web sites . they empirically compare alternative methods and publish benchmark data sets . |
| Outcome: | The proposed methods improve state-of-the-art results on common benchmarks, the authors show . the pipeline has been tested on Russian, Sinhala, Nepali, Tagalog, Swahili, and Somali . |
Similar Papers
JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent machine translation algorithms rely on parallel corpora, but only some resource-rich language pairs can benefit from them. |
| Approach: | They construct a parallel corpus for English-Japanese, which has 8.7 million sentence pairs . they use a web crawler to automatically align parallel sentences in the corpus . |
| Outcome: | The proposed corpus includes a broader range of domains and can be trained with a pre-trained model. |
Document-Level Machine Translation with Large-Scale Public Parallel Corpora (2024.acl-long)
Copied to clipboard
| Challenge: | Document-level machine translation has inherent advantages over sentence-level translation due to additional information available to a model from document context. |
| Approach: | They propose to use document context to train context-aware models on these datasets and to use it to model document-level phenomena. |
| Outcome: | The proposed models improve translation quality and target document-level phenomena by incorporating contextual information from several preceding sentences. |
JParaCrawl v3.0: A Large-scale English-Japanese Parallel Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing parallel corpora for English-Japanese are limited, limiting the accuracy of machine translation models. |
| Approach: | They propose a web-based English-Japanese parallel corpus with 21 million unique sentence pairs . this is more than twice as many as the previous corpus JParaCrawl v2.0 . |
| Outcome: | The proposed corpus boosts the accuracy of machine translation models on various domains. |
Automatic Creation of Text Corpora for Low-Resource Languages from the Internet: The Case of Swiss German (2020.lrec-1)
Copied to clipboard
| Challenge: | Despite the small pool of speakers, there are still few natural language processing corpora, studies or tools for Swiss German. |
| Approach: | They propose to use a web scraper to generate the largest Swiss German text corpus . they show that the tool can be applied to other low-resource languages as well . |
| Outcome: | The proposed tool significantly improves language modeling in Swiss German, the authors show . |
Building a Web-Scale Dependency-Parsed Corpus from CommonCrawl (L18-1)
Copied to clipboard
| Challenge: | DepCC is the largest-to-date linguistically analyzed corpus in English . large corpora are essential for the modern data-driven approaches to natural language processing . |
| Approach: | They present a large-to-date linguistically analyzed corpus in English with 365 million documents . they build an index of all sentences and their linguistic meta-data enabling quick search across the corpus . |
| Outcome: | The proposed model outperforms state-of-the-art models on smaller corpora on the SimVerb3500 dataset. |
Exploring Paracrawl for Document-level Neural Machine Translation (2023.eacl-main)
Copied to clipboard
| Challenge: | Document-level neural machine translation (NMT) has outperformed sentence-level NMT on a number of datasets. |
| Approach: | They use Paracrawl to extract parallel paragraphs from Paracral webpages . they also use the extracted parallel paragraph as parallel documents for training . |
| Outcome: | The proposed model outperforms sentence-level NMT on a number of datasets. |
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021.emnlp-main)
Copied to clipboard
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner
| Challenge: | Large text corpora are often introduced with minimal documentation . documenting collection process, composition, intended uses, and other are key for structured, task-specific datasets. |
| Approach: | They propose to document a dataset created by applying filters to a single snapshot of Common Crawl. |
| Outcome: | The proposed dataset shows that blocklist filtering removes text from minority individuals and patents. |
A New Massive Multilingual Dataset for High-Performance Language Technologies (2024.lrec-main)
Copied to clipboard
Ona de Gibert, Graeme Nail, Nikolay Arefyev, Marta Bañón, Jelmer van der Linde, Shaoxiong Ji, Jaume Zaragoza-Bernabeu, Mikko Aulamo, Gema Ramírez-Sánchez, Andrey Kutuzov, Sampo Pyysalo, Stephan Oepen, Jörg Tiedemann
| Challenge: | a new massive multilingual dataset is available for language modeling and machine translation training. |
| Approach: | They present a massive multilingual dataset using web crawls from the Internet Archive and CommonCrawl . they use open-source software tools and high-performance computing to acquire, manage and process large corpora . |
| Outcome: | The HPLT language resources is a massive multilingual dataset . it includes monolingual and bilingual corpora extracted from CommonCrawl and the Internet Archive . the results are published online at the journal journal cense4 . |
Towards a Cleaner Document-Oriented Multilingual Crawled Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | Existing web crawling pipelines are used to collect large corpora raw data, but the main way to collect such data is through manual data extraction. |
| Approach: | They propose to use a web crawler to extract and classify data from a multilingual web corpus and an automated annotation pipeline to improve it. |
| Outcome: | The proposed version of OSCAR could be used to pre-train large generative language models and other applications in Natural Language Processing and Digital Humanities. |
Trafilatura: A Web Scraping Library and Command-Line Tool for Text Discovery and Extraction (2021.acl-demo)
Copied to clipboard
| Challenge: | Existing tools for text extraction and web corpus construction are not enough to extract and pre-process web data to meet scientific expectations with respect to text quality. |
| Approach: | They propose a text discovery and extraction tool published under open-source license that allows for main text, comments and metadata extraction while also providing building blocks for web crawling tasks. |
| Outcome: | The proposed tool performs significantly better than other open-source solutions on real-world data and in external benchmarks. |