BERTSeg: BERT Based Unsupervised Subword Segmentation for Neural Machine Translation (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing subword segmenters are frequency-based without semantics information or neural-based but trained on parallel corpora. |
| Approach: | They propose an unsupervised neural subword segmenter for neural machine translation that utilizes contextualized semantic embeddings of words from characterBERT and maximizes the generation probability of subword segments. |
| Outcome: | The proposed method improves translation performance on ALT, IWSLT15 Vi->En, WMT16 Ro->En and WMT15 Fi->En datasets. |
Similar Papers
Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates (P18-1)
Copied to clipboard
| Challenge: | Subword units are an effective way to alleviate the open vocabulary problems in neural machine translation. |
| Approach: | They propose a method to regularize subword segmentations probabilistically by sampling subwords . they also propose 'unigram' language model to be used for better subword sampling . |
| Outcome: | The proposed method improves on low resource and out-of-domain settings with multiple corpora. |
Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation (2023.findings-acl)
Copied to clipboard
| Challenge: | Subword segmenters are used in neural machine translation, but are not used in high-resource settings. |
| Approach: | They propose a subword segmental machine translation (SSMT) that unifies subword and MT in a single trainable model. |
| Outcome: | The proposed model improves chrF scores for morphologically rich agglutinative languages and is more robust on a test set constructed for evaluating morphology generalisations. |
Bilingual Subword Segmentation for Neural Machine Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing subword segmentation methods tokenize sentences without considering translation . proposed method could be more favorable to machine translation if it uses bilingual sentences . |
| Approach: | They propose a subword segmentation method that tokenizes sentences by using subword units induced from bilingual sentences. |
| Outcome: | The proposed method improves translation performance on translation tasks up to +0.81 BLEU. |
Probabilistic Bilingual Subword Segmentation with Latent Subword Alignment (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing methods do not consider parallel relationships, preventing translation model training. |
| Approach: | They propose a method for learning subword correspondences in parallel sentence pairs using the EM algorithm. |
| Outcome: | The proposed method improves translation accuracy for many tasks. |
Unsupervised Chinese Word Segmentation with BERT Oriented Probing and Transformation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for unsupervised Chinese word segmentation exploit shallow semantic information, which can miss important context. |
| Approach: | They propose to take advantage of deep contextual semantic information with a self-training manner to transform it into explicit word segmentation ability. |
| Outcome: | The proposed approach achieves state-of-the-art F1 score on two CWS benchmark datasets. |
An Effective Approach to Unsupervised Machine Translation (P19-1)
Copied to clipboard
| Challenge: | a recent research line has managed to train both unsupervised and unsupervised machine translation systems using monolingual corpora only. |
| Approach: | They propose to use monolingual corpora to train both unsupervised and unsupervised machine translation systems. |
| Outcome: | The proposed system achieves 22.5 BLEU points in English-to-German WMT 2014, 5.5 points more than the previous best unsupervised system, and 0.5 points more in the (supervised) shared task winner back in 2014. |
When is Char Better Than Subword: A Systematic Study of Segmentation Algorithms for Neural Machine Translation (2021.acl-short)
Copied to clipboard
| Challenge: | Subword segmentation algorithms can produce sub-optimal segmentation when the target language is rich in morphological changes or there is not enough data for learning compact composition rules. |
| Approach: | They compare character-based and subword-based neural machine translation systems . they find character-driven models are better at handling morphological phenomena . |
| Outcome: | The character-based models are better at handling morphological phenomena, generating rare and unknown words, and more suitable for transferring to unseen domains. |
Adversarial Subword Regularization for Robust Neural Machine Translation (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for segmenting words into subword units are not robust enough to handle multiple subword candidates. |
| Approach: | They propose to regularize subword segmentations that maximize the translation loss by using gradient signals during training to prevent erroneous segmentations of unseen words. |
| Outcome: | The proposed method improves the performance of NMT models on low-resource and out-domain datasets. |
Single Model Ensemble for Subword Regularized Models in Low-Resource Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing subword regularizations use multiple segmentations during training but only use one segmentation in inference. |
| Approach: | They propose an inference strategy that uses multiple subword segmentations to solve this discrepancy in the training process and inference. |
| Outcome: | The proposed strategy reduces the cost of training and improves the performance of models trained with subword regularization in low-resource machine translation tasks. |
ReWE: Regressing Word Embeddings for Regularization of Neural Machine Translation Systems (N19-1)
Copied to clipboard
| Challenge: | Existing methods to regularize neural machine translation are limited in low-resource settings. |
| Approach: | They propose a method that uses regressing word embeddings to regularize neural machine translation. |
| Outcome: | The proposed system improves on a strong baseline and a state-of-the-art system. |