| Challenge: | Existing methods to filter out noisy parallel sentences from web crawled data are in demand. |
| Approach: | They propose a method to filter out noisy sentence pairs from web crawled corpora using pre-trained language models. |
| Outcome: | The proposed method outperforms baselines and achieves state-of-the-art on two datasets. |
Similar Papers
Investigating Web Corpus Filtering Methods for Language Model Development in Japanese (2024.naacl-srw)
Copied to clipboard
| Challenge: | a high quality web corpus is essential for large language models to be developed . strong filtering methods can lead to lesser performance in downstream tasks . |
| Approach: | They build classifiers and language models that can process large amounts of corpora quickly enough for pretraining LLMs. |
| Outcome: | The proposed method is the most accurate and leads to lesser performance in downstream tasks. |
Improving Machine Translation with Phrase Pair Injection and Corpus Filtering (2022.emnlp-main)
Copied to clipboard
| Challenge: | In this paper, we show that the combination of Phrase Pair Injection and Corpus Filtering boosts the performance of Neural Machine Translation systems. |
| Approach: | They propose to combine Phrase Pair Injection and Corpus Filtering to boost performance of Neural Machine Translation systems. |
| Outcome: | The proposed method improves machine translation models on low-resource language pairs . BLEU score improves over models trained with whole pseudo-parallel corpus augmented with parallel corpus. |
Filtering and Mining Parallel Data in a Joint Multilingual Space (P18-2)
Copied to clipboard
| Challenge: | Using a cosine distance in a joint multilingual sentence embedding, we filter out noisy parallel data and mine for bitexts in large news collections. |
| Approach: | They propose to learn a joint multilingual sentence embedding and use the distance between sentences in different languages to filter noisy parallel data and to mine for parallel data in large monolingual texts. |
| Outcome: | The proposed approach improves a competitive baseline on the WMT'14 task by 0.3 BLEU by filtering out 25% of the training data. |
Phrase-Based & Neural Unsupervised Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Recent advances in machine translation have reported near human-level performance on several languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences. |
| Approach: | They propose two models that leverage a careful initialization of the parameters and denoising effect of language models. |
| Outcome: | The proposed models outperform the current methods on English-French and German-English benchmarks while being simpler and having fewer hyper-parameters. |
Multilingual Data Filtering using Synthetic Data from Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that effective filters can be created by utilising Large Language Models to synthetically label data, which is then used to train smaller neural models for filtering purposes. |
| Approach: | They extend this approach to languages beyond English to train neural models for filtering purposes. |
| Outcome: | The proposed approach is effective at filtering parallel text for translation quality and filtering for domain specificity. |
A Recipe of Parallel Corpora Exploitation for Multilingual Large Language Models (2025.findings-naacl)
Copied to clipboard
| Challenge: | Recent studies have highlighted the potential of exploiting parallel corpora to enhance multilingual large language models. |
| Approach: | They investigate the impact of parallel corpora quality and quantity, training objectives, and model size on performance of multilingual large language models enhanced with parallel corporeal. |
| Outcome: | The proposed approach improves performance in bilingual and general-purpose tasks. |
Multilingual Translation via Grafting Pre-trained Language Models (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to graft pre-trained (masked) language models to multilingual data are limited, and they lack cross-attention component. |
| Approach: | They propose to graft separately pre-trained (masked) language models for machine translation using monolingual data and parallel data. |
| Outcome: | The proposed method achieves average improvements of 5.8 BLEU in x2en and 2.9 BLUE in en2x directions compared with the multilingual Transformer of the same size. |
Fixing Translation Divergences in Parallel Corpora for Neural MT (D18-1)
Copied to clipboard
| Challenge: | Existing methods to detect translation divergences from parallel corpora are noisy and limited in size. |
| Approach: | They propose an unsupervised method for detecting translation divergences in parallel sentences . they use a neural network that computes cross-lingual sentence similarity scores . |
| Outcome: | The proposed method improves translation performance for English-French and English-German translation tasks. |
Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings (P19-1)
Copied to clipboard
| Challenge: | Traditional parallel corpus mining methods focus on the textual content instead of the size and quality of training data. |
| Approach: | They propose a method for machine translation based on multilingual sentence embeddings. |
| Outcome: | The proposed method outperforms the best published methods on the BUCC mining task and the UN reconstruction task by more than 10 F1 and 30 precision points. |
JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Recent machine translation algorithms rely on parallel corpora, but only some resource-rich language pairs can benefit from them. |
| Approach: | They construct a parallel corpus for English-Japanese, which has 8.7 million sentence pairs . they use a web crawler to automatically align parallel sentences in the corpus . |
| Outcome: | The proposed corpus includes a broader range of domains and can be trained with a pre-trained model. |