| Challenge: | Existing word alignment models are not accurate for word alignments. |
| Approach: | They propose a method to train a Transformer model to produce accurate translations and alignments. |
| Outcome: | The proposed model outperforms GIZA++ trained models on translation and alignment tasks while maintaining translation accuracy. |
Similar Papers
End-to-End Neural Word Alignment Outperforms GIZA++ (2020.acl-main)
Copied to clipboard
| Challenge: | Word alignment was once a core unsupervised learning task in natural language processing . but word alignment still plays an important role in interactive applications of neural machine translation, such as annotation transfer and lexicon injection. |
| Approach: | They propose to use a Transformer model to train an unsupervised word alignment model. |
| Outcome: | The proposed method outperforms GIZA++ on three data sets and is tightly integrated and does not affect translation quality. |
A Bidirectional Transformer Based Alignment Model for Unsupervised Word Alignment (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning word alignment include statistical word aligners (e.g. GIZA++) Existing word alignment models employ a target-to-source attention mechanism which can provide rough word alignments but with a low accuracy. |
| Approach: | They propose a bidirectional Transformer based alignment model for unsupervised learning of the word alignment task. |
| Outcome: | The proposed model outperforms both previous neural word alignment approaches and the popular statistical word aligner GIZA++ on three word alignment tasks. |
A Discriminative Neural Model for Cross-Lingual Word Alignment (D19-1)
Copied to clipboard
| Challenge: | a novel word alignment model for machine translation has been developed for a number of languages . explicit word-to-word alignments have largely been lost in neural MT systems . |
| Approach: | They propose a discriminative word alignment model which integrates into a Transformer-based machine translation model. |
| Outcome: | The proposed model performs better on Chinese and Arabic alignments than standard models. |
Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing pre-training methods are not effective for machine translation tasks. |
| Approach: | They propose a method to pre-train a universal multilingual neural machine translation model . they use random aligned substitution technique to bring words and phrases with similar meanings closer in the representation space. |
| Outcome: | The proposed approach improves translation quality on low, medium, rich resource languages. |
Accurate Word Alignment Induction from Neural Machine Translation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Prior work suggests that Transformer captures poor word alignments through its attention mechanism. |
| Approach: | They propose two new word alignment induction methods that use attention weights to capture accurate word alignments. |
| Outcome: | The proposed methods outperform baselines on three publicly available datasets and are significantly better than GIZA++. |
Improving Word Alignment Using Semi-Supervised Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing word alignment methods rely on labeled data, but augmenting training with pseudo-labeled data improves performance. |
| Approach: | They propose a semi-supervised framework to improve word alignment methods . they use pseudo-labeled data from multilingual encoder models as word aligners . |
| Outcome: | The proposed framework outperforms the current state-of-the-art binary alignment method on word alignment datasets. |
Rethinking Document-level Neural Machine Translation (2022.findings-acl)
Copied to clipboard
| Challenge: | Neural machine translation models are weak enough for document-level translation . current models only translate sentences individually, resulting in poor document coherence . |
| Approach: | They propose to use the original Transformer model to test document-level neural machine translation . they find that the original transformer models can achieve strong results for document translation if trained properly . |
| Outcome: | The proposed model outperforms sentence-level models on nine datasets and two sentence- level datasets across six languages. |
Aligners: Decoupling LLMs and Alignment (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) need to be aligned with human expectations to ensure their safety and utility in most applications. |
| Approach: | They propose to decouple LLMs and alignment by training *aligner* models that can be used to align any LLM on an as-needed basis. |
| Outcome: | The proposed model can be used to align any LLM for a given criteria on an as-needed basis. |
On the Word Alignment from Neural Machine Translation (P19-1)
Copied to clipboard
| Challenge: | Prior researches suggest that neural machine translation (NMT) captures word alignment through its attention mechanism, however, attention may fail to capture word alignment for some NMT models. |
| Approach: | They propose two methods to induce word alignment which are general and agnostic to specific NMT models. |
| Outcome: | The proposed methods induce much better word alignment than attention. |
TransAlign: Machine Translation Encoders are Strong Word Aligners, Too (2025.findings-emnlp)
Copied to clipboard
| Challenge: | translation-based approaches to cross-lingual transfer (XLT) are limited. |
| Approach: | They propose a word aligner that utilizes the encoder of a massively multilingual MT model. |
| Outcome: | The proposed word aligner outperforms existing WA and state-of-the-art non-WA-based methods in token classification tasks. |