A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning word alignment include statistical word aligners (e.g. GIZA++) Existing word alignment models employ a target-to-source attention mechanism which can provide rough word alignments but with a low accuracy. |
| Approach: | They propose a bidirectional Transformer based alignment model for unsupervised learning of the word alignment task. |
| Outcome: | The proposed model outperforms both previous neural word alignment approaches and the popular statistical word aligner GIZA++ on three word alignment tasks. |
Similar Papers
Jointly Learning to Align and Translate with Transformer Models (D19-1)
Copied to clipboard
| Challenge: | Existing word alignment models are not accurate for word alignments. |
| Approach: | They propose a method to train a Transformer model to produce accurate translations and alignments. |
| Outcome: | The proposed model outperforms GIZA++ trained models on translation and alignment tasks while maintaining translation accuracy. |
End-to-End Neural Word Alignment Outperforms GIZA++ (2020.acl-main)
Copied to clipboard
| Challenge: | Word alignment was once a core unsupervised learning task in natural language processing . but word alignment still plays an important role in interactive applications of neural machine translation, such as annotation transfer and lexicon injection. |
| Approach: | They propose to use a Transformer model to train an unsupervised word alignment model. |
| Outcome: | The proposed method outperforms GIZA++ on three data sets and is tightly integrated and does not affect translation quality. |
A Discriminative Neural Model for Cross-Lingual Word Alignment (D19-1)
Copied to clipboard
| Challenge: | a novel word alignment model for machine translation has been developed for a number of languages . explicit word-to-word alignments have largely been lost in neural MT systems . |
| Approach: | They propose a discriminative word alignment model which integrates into a Transformer-based machine translation model. |
| Outcome: | The proposed model performs better on Chinese and Arabic alignments than standard models. |
Mask-Align: Self-Supervised Neural Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Word alignment is an important task in many natural language processing tasks. |
| Approach: | They propose a self-supervised word alignment model that takes advantage of the full context on the target side. |
| Outcome: | The proposed model outperforms previous unsupervised models and obtains state-of-the-art results on four language pairs. |
Accurate Word Alignment Induction from Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work suggests that Transformer captures poor word alignments through its attention mechanism. |
| Approach: | They propose two new word alignment induction methods that use attention weights to capture accurate word alignments. |
| Outcome: | The proposed methods outperform baselines on three publicly available datasets and are significantly better than GIZA++. |
Third-Party Aligner for Neural Word Alignments (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work shows that word alignment can be competitive . |
| Approach: | They propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. |
| Outcome: | The proposed approach can find more accurate word alignments and delete wrong alignments, leading to better performance than the current best third-party word aligner. |
Assessing Non-autoregressive Alignment in Neural Machine Translation via Word Reordering (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing non-autoregressive neural machine translation models that implicitly model dependencies are sub-optimal in handling word order errors. |
| Approach: | They propose to learn a non-autoregressive language model that can be combined with Viterbi decoding to achieve better reordering performance. |
| Outcome: | The proposed model outperforms state-of-the-art reordering mechanisms under different word permutation settings with a 2-27 BLEU improvement, suggesting high potential for word alignment in NAT. |
On the Word Alignment from Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Prior researches suggest that neural machine translation (NMT) captures word alignment through its attention mechanism, however, attention may fail to capture word alignment for some NMT models. |
| Approach: | They propose two methods to induce word alignment which are general and agnostic to specific NMT models. |
| Outcome: | The proposed methods induce much better word alignment than attention. |
Constraining word alignments with posterior regularization for label transfer (2022.naacl-industry)
Copied to clipboard
| Challenge: | Unsupervised word alignments are not always possible in industrial NLP pipelines, where multilingual annotation guidelines are complex and deviate from semantic consistency due to various factors. |
| Approach: | They propose to constrain word alignment models to remain consistent with both source and target annotation guidelines by leveraging posterior regularization and labeled examples. |
| Outcome: | The proposed model improves on the multiATIS++ dataset over AWESoME, and even a small amount of target language annotations can help. |
Improving Word Alignment Using Semi-Supervised Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing word alignment methods rely on labeled data, but augmenting training with pseudo-labeled data improves performance. |
| Approach: | They propose a semi-supervised framework to improve word alignment methods . they use pseudo-labeled data from multilingual encoder models as word aligners . |
| Outcome: | The proposed framework outperforms the current state-of-the-art binary alignment method on word alignment datasets. |