| Challenge: | State-of-the-art word alignment training methods require a different class depending on the availability of gold data for a particular language pair. |
| Approach: | They propose a novel word alignment technique based on binary sequence labeling that outperforms existing approaches in both scenarios. |
| Outcome: | The proposed method outperforms existing models on non-English language pairs and performs stratified error analysis over alignment error type. |
Similar Papers
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
SilverAlign: MT-Based Silver Data Algorithm for Evaluating Word Alignment (2024.lrec-main)
Copied to clipboard
| Challenge: | Word alignments are crucial for a variety of NLP tasks. |
| Approach: | They propose a method to automatically create silver data for evaluation of word aligners by exploiting machine translation and minimal pairs. |
| Outcome: | The proposed method correlates with gold benchmarks for 9 language pairs, making it a valid resource for evaluation of different languages and domains when gold data is not available. |
Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality. |
| Approach: | They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model. |
| Outcome: | The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs. |
Meeting the Needs of Low-Resource Languages: The Value of Automatic Alignments via Pretrained Models (2023.eacl-main)
Copied to clipboard
Abteen Ebrahimi, Arya D. McCarthy, Arturo Oncevay, John E. Ortega, Luis Chiruzzo, Gustavo Giménez-Lugo, Rolando Coto-Solano, Katharina Kann
| Challenge: | Large multilingual models have inspired a new class of word alignment methods, which work well for pretraining languages. |
| Approach: | They propose to use transformer-based word alignment methods to extract alignments from massive pretrained models. |
| Outcome: | The proposed methods outperform traditional methods for languages unseen to pretraining models, and are competitive with each other. |
Improving Word Alignment Using Semi-Supervised Learning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing word alignment methods rely on labeled data, but augmenting training with pseudo-labeled data improves performance. |
| Approach: | They propose a semi-supervised framework to improve word alignment methods . they use pseudo-labeled data from multilingual encoder models as word aligners . |
| Outcome: | The proposed framework outperforms the current state-of-the-art binary alignment method on word alignment datasets. |
PreAlign: Boosting Cross-Lingual Transfer by Early Establishment of Multilingual Alignment (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large language models exhibit reasonable multilingual abilities, despite predominantly English-centric pretraining. |
| Approach: | They propose a framework that establishes multilingual alignment prior to language model pretraining and preserves this alignment using a code-switching strategy during pretraining. |
| Outcome: | Experiments in a synthetic English to English-Clone setting show that PreAlign outperforms standard multilingual joint training in language modeling, zero-shot cross-lingual transfer, and cross-linguistic knowledge application. |
Deep Generative Model for Joint Alignment and Word Representation (N18-1)
Copied to clipboard
| Challenge: | EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. |
| Approach: | They exploit translation as a distributional context and embed words as posterior probability densities, rather than point estimates, which allows them to compare words in context using a measure of overlap between distributions. |
| Outcome: | The proposed model performs on a range of lexical semantics tasks and achieves competitive results on benchmarks including natural language inference, paraphrasing, and text similarity. |
MetaAlign: Align Large Language Models with Diverse Preferences during Inference Time (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to align large language models with human preferences often result in a static alignment that cannot account for the diversity of human preferences in practical applications. |
| Approach: | They propose a method to help large language models dynamically align with various explicit or implicit preferences specified at inference time. |
| Outcome: | The proposed method can help LLMs dynamically align with various explicit or implicit preferences specified at the inference stage, validating the feasibility of MetaAlign. |
WSPAlign: Word Alignment Pre-training via Large-Scale Weakly Supervised Span Prediction (2023.acl-long)
Copied to clipboard
| Challenge: | Existing word alignment methods rely on manual data and lack generalization ability. |
| Approach: | They propose to use a weakly-supervised large-scale weakly supervised dataset for word alignment pre-training via span prediction to reduce the need for manual data. |
| Outcome: | The proposed method improves upon the best supervised baseline by 3.3 6.1 points in F1 and 1.5 6.1 point in AER. |
More Embeddings, Better Sequence Labelers? (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work suggests contextual embeddings improve sequence labeling accuracy . but, there is no definite conclusion on whether concatenating different kinds of embeddables is effective . |
| Approach: | They propose a family of contextual embeddings that improves sequence labeling accuracy . they conduct extensive experiments on 3 tasks over 18 datasets and 8 languages . |
| Outcome: | The proposed family of contextual embeddings improves the accuracy of sequence labelers over non-contextual embedders. |