| Challenge: | Existing speech-to-speech parallel data is scarce and expensive to create from scratch. |
| Approach: | They propose an algorithm which automatically aligns pairs of speech segments aligned in meaning and expressivity. |
| Outcome: | The proposed algorithm outperforms semantic-focused approaches on content translation quality. |
Similar Papers
Cross-lingual AMR Aligner: Paying Attention to Cross-Attention (2023.findings-acl)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) graphs embed the semantics of a sentence in a directed acyclic graph, where concepts are represented by nodes, semantic relations between concepts by edges, and the co-references by reentrant nodes. |
| Approach: | They propose a novel aligner for Abstract Meaning Representation graphs that scales cross-lingually and can align units and spans in sentences of different languages. |
| Outcome: | The proposed aligner achieves state-of-the-art in the benchmarks and can scale cross-lingually. |
Aligning Multilingual Embeddings for Improved Code-switched Natural Language Understanding (2022.coling-1)
Copied to clipboard
| Challenge: | a recent study has shown that multilingual models can be effective on monolingual data but need additional training to work well with code-switched text. |
| Approach: | They propose to train multilingual models with alignment objectives using parallel text . they find such an explicit alignment step improves performance on code-switched NLP tasks . |
| Outcome: | The proposed model improves on Hindi-English Sentiment Analysis, Named Entity Recognition and Question Answering tasks. |
Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)
Copied to clipboard
| Challenge: | Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models. |
| Approach: | They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key . |
| Outcome: | The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key . |
Alignment Data base for a Sign Language Concordancer (2020.lrec-1)
Copied to clipboard
| Challenge: | a new study examines the need for sign language translators to have tools similar to text-to-text translation. |
| Approach: | They propose to use a concordancer to search for parallel Franch-LSF segments . they use dozens of short news clips and 120 SL videos to align them manually . |
| Outcome: | The proposed data base will be searched using a concordancer and expand in the future. |
Take a Closer Look at Multilinguality! Improve Multilingual Pre-Training Using Monolingual Corpora Only (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have demonstrated remarkable cross-lingual capability of pre-trained language models . however, semantic alignments may be the reason behind such capability but remain under-explored. |
| Approach: | They propose token-level and semantic-level code-switched masked language modeling to improve cross-lingual interactions over mono-mPLMs without parallel sentences. |
| Outcome: | The proposed method outperforms mono-mPLMs on natural language understanding and unsupervised machine translation tasks. |
Multi-level Alignment Pretraining for Multi-lingual Semantic Parsing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for multilingual semantic parsing only handle monolingual parsers, while in real world applications such as Chatbot and search engine, we generally need to handle multi-lingual semanticparsing. |
| Approach: | They propose a multi-level alignment pretraining method in a unified architecture for multi-lingual semantic parsing. |
| Outcome: | The proposed method outperforms state-of-the-art methods on a publicly avail-able multi-lingual semantic parsing dataset and a newly constructed dataset. |
A Multimodal French Corpus of Aligned Speech, Text, and Pictogram Sequences for Speech-to-Pictogram Machine Translation (2024.lrec-main)
Copied to clipboard
Cécile Macaire, Chloé Dion, Jordan Arrigo, Claire Lemaire, Emmanuelle Esperança-Rodier, Benjamin Lecouteux, Didier Schwab
| Challenge: | Existing algorithms for the automatic translation of spoken language into pictogram units are lacking for language impairments. |
| Approach: | They propose to use a French dataset that contains 230 hours of speech resources to create a rule-based pictogram grammar with a restricted vocabulary and a discussion of strategic decisions. |
| Outcome: | The proposed model is validated through multiple post-editing phases by expert annotators and is freely available under a non-commercial licence. |
Inducing and Using Alignments for Transition-based AMR Parsing (2022.naacl-main)
Copied to clipboard
Andrew Drozdov, Jiawei Zhou, Radu Florian, Andrew McCallum, Tahira Naseem, Yoon Kim, Ramón Astudillo
| Challenge: | Abstract Meaning Representation parsers rely on node-to-word alignments, but lack the complexity of the pipeline. |
| Approach: | They propose a neural aligner for abstract meaning representation that learns node-to-word alignments without relying on pipelines. |
| Outcome: | The proposed approach improves accuracy and generalization from AMR2.0 to AMR3.0 corpora. |
Speech Vecalign: an Embedding-based Method for Aligning Parallel Speech Documents (2025.emnlp-main)
Copied to clipboard
| Challenge: | Speech Vecalign is a parallel speech document alignment method that monotonically aligns speech segment embeddings and does not depend on text transcriptions. |
| Approach: | They propose a parallel speech document alignment method that monotonically aligns speech segment embeddings and does not depend on text transcriptions. |
| Outcome: | The proposed method outperforms SpeechMatrix models on 3,000 hours of unlabeled speech documents and produces longer speech-to-speech alignments. |
SimAlign: High Quality Word Alignments Without Parallel Training Data Using Static and Contextualized Embeddings (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Word alignments are useful for statistical and neural machine translation (NMT) and cross-lingual annotation projection. |
| Approach: | They propose to leverage multilingual word embeddings for word alignment. |
| Outcome: | The proposed methods perform better for four languages and comparable for two languages than traditional statistical aligners even with abundant parallel data. |