Speech Vecalign: an Embedding-based Method for Aligning Parallel Speech Documents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Speech Vecalign is a parallel speech document alignment method that monotonically aligns speech segment embeddings and does not depend on text transcriptions. |
| Approach: | They propose a parallel speech document alignment method that monotonically aligns speech segment embeddings and does not depend on text transcriptions. |
| Outcome: | The proposed method outperforms SpeechMatrix models on 3,000 hours of unlabeled speech documents and produces longer speech-to-speech alignments. |
Similar Papers
Vecalign: Improved Sentence Alignment in Linear Time and Space (D19-1)
Copied to clipboard
| Challenge: | Sentence-aligned bitext is used to train nearly all machine translation systems. |
| Approach: | They propose a bilingual sentence alignment method which is linear in time and space with respect to the number of sentences being aligned. |
| Outcome: | The proposed method outperforms the existing method by 5 F1 points on a German–French test set and improves downstream MT quality by 1.7 and 1.6 BLEU in Sinhala-English and Nepali-English, respectively. |
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations (2023.acl-long)
Copied to clipboard
Paul-Ambroise Duquenne, Hongyu Gong, Ning Dong, Jingfei Du, Ann Lee, Vedanuj Goswami, Changhan Wang, Juan Pino, Benoît Sagot, Holger Schwenk
| Challenge: | SpeechMatrix is a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. |
| Approach: | They present a large-scale multilingual corpus of speech-to-speech translations mined from real speech of European Parliament recordings. |
| Outcome: | The proposed model can train bilingual models on 136 language pairs with 418 thousand hours of speech. |
SimAlign: High Quality Word Alignments Without Parallel Training Data Using Static and Contextualized Embeddings (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Word alignments are useful for statistical and neural machine translation (NMT) and cross-lingual annotation projection. |
| Approach: | They propose to leverage multilingual word embeddings for word alignment. |
| Outcome: | The proposed methods perform better for four languages and comparable for two languages than traditional statistical aligners even with abundant parallel data. |
SpeechAlign: A Framework for Speech Translation Alignment Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Speech-to-Speech and Speech- to-Text translation are currently dynamic areas of research. |
| Approach: | They propose a framework to evaluate source-target alignment in speech models . they introduce a speech gold alignment dataset and introduce two new metrics . |
| Outcome: | The proposed framework evaluates source-target alignment quality within speech models. |
Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality. |
| Approach: | They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model. |
| Outcome: | The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs. |
Neuralign: A Context-Aware, Cross-Lingual and Fully-Neural Sentence Alignment System for Long Texts (2024.eacl-long)
Copied to clipboard
| Challenge: | Existing sentence alignment systems focus on auxiliary information such as document metadata and hyperparameter-sensitive techniques, and neglect the crucial role that context plays in the alignment process. |
| Approach: | They propose a context-aware, end-to-end and fully-neural architecture for sentence alignment that maps source and target sentences in long documents by contextualizing their sentence embeddings with respect to the other sentences in the document. |
| Outcome: | The proposed system maps source and target sentences in long documents by contextualizing their sentence embeddings with respect to the other sentences in the document. |
Deep Generative Model for Joint Alignment and Word Representation (N18-1)
Copied to clipboard
| Challenge: | EmbedAlign model embeds words in their complete observed context and learns by marginalisation of latent lexical alignments. |
| Approach: | They exploit translation as a distributional context and embed words as posterior probability densities, rather than point estimates, which allows them to compare words in context using a measure of overlap between distributions. |
| Outcome: | The proposed model performs on a range of lexical semantics tasks and achieves competitive results on benchmarks including natural language inference, paraphrasing, and text similarity. |
Segment, Embed, and Align: A Universal Recipe for Aligning Subtitles to Signing (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches for aligning spoken language text to sign language videos rely on end-to-end training tied to a specific language or dataset. |
| Approach: | They propose a universal approach for aligning spoken language text with corresponding timestamps to sign language videos using a lightweight dynamic programming procedure. |
| Outcome: | The proposed method can be used on four sign language datasets and is highly efficient on CPU. |
SentAlign: Accurate and Scalable Sentence Alignment (2023.emnlp-demo)
Copied to clipboard
| Challenge: | SentAlign is an automatic sentence alignment tool designed for large documents . it evaluates all possible alignment paths in documents of thousands of sentences . |
| Approach: | They present a sentence alignment tool that evaluates all possible alignment paths in parallel documents of thousands of sentences and uses a divide-and-conquer approach to align documents containing tens of thousands. |
| Outcome: | The proposed tool outperforms five other sentence alignment tools on two evaluation sets and on a downstream machine translation task. |
Word Alignment by Fine-tuning Embeddings on Parallel Corpora (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing work on word alignment has focused on unsupervised learning on parallel text. |
| Approach: | They propose to combine pre-trained contextualized word embeddings with multilingually trained language models to achieve competitive results on word alignment tasks. |
| Outcome: | The proposed model outperforms state-of-the-art models on five language pairs and can train multilingual word aligners that can obtain robust performance on different language pairs. |