Javier Iranzo-Sánchez, Adrià Giménez Pastor, Joan Albert Silvestre-Cerdà, Pau Baquero-Arnal, Jorge Civera Saiz, Alfons Juan
| Challenge: | Existing approaches to stream ST combine advances in ASR and MT to achieve high quality translations without compromising the speed of the system. |
| Approach: | They propose to concatenate an Automatic Speech Recognition system followed by a Machine Translation system. |
| Outcome: | The proposed models improve on the Europarl-ST dataset on the BLEU score. |
Similar Papers
Streaming Models for Joint Speech Recognition and Translation (2021.eacl-main)
Copied to clipboard
| Challenge: | Using end-to-end models for speech translation has become a focus of the ST community . cascaded models have the advantage of including automatic speech recognition output . |
| Approach: | They propose a model that condenses sound waves into translated text and integrates automatic speech recognition outputs into the models. |
| Outcome: | The proposed model is statistically similar to cascading models, but has half the number of parameters. |
CLAD-ST: Contrastive Learning with Adversarial Data for Robust Speech Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Cascaded approach is the most popular choice for speech translation, but lacks robustness when dealing with noisy inputs. |
| Approach: | They propose a cascaded approach that uses an automatic speech recognition model and a machine translation model to translate speech in one language to text in another language. |
| Outcome: | The proposed approach achieves significant gains of up to 3 BLEU scores in English-German and English-French speech translation without hurting the translation quality on clean text. |
Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders (2021.acl-long)
Copied to clipboard
| Challenge: | End-to-end Speech Translation (E2E ST) encoders lack global context representation, whereas MT encoder lacks it. |
| Approach: | They propose a Stacked Acoustic-and-Textual Encoding method for speech translation . they propose an adaptor module to alleviate representation inconsistency . |
| Outcome: | The proposed method achieves state-of-the-art BLEU scores of 18.3 and 25.2 on two ST tasks. |
Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference? (2021.acl-long)
Copied to clipboard
Luisa Bentivogli, Mauro Cettolo, Marco Gaido, Alina Karakanta, Alberto Martinelli, Matteo Negri, Marco Turchi
| Challenge: | a gap between direct approaches to speech translation (ST) and traditional cascade solutions has gradually decreased . a recent study found that the subtle differences observed in their behavior are not sufficient for humans neither to distinguish them nor to prefer one over the other. |
| Approach: | They compare state-of-the-art systems representative of the two paradigms . they find subtle differences observed in their behavior are not sufficient . |
| Outcome: | The proposed system is compared with state-of-the-art systems representative of the two paradigms. |
Phone Features Improve Speech Translation (2020.acl-main)
Copied to clipboard
| Challenge: | End-to-end models for speech translation more tightly couple speech recognition (ASR) and machine translation (MT) compared to cascades, but performance gap remains in low-resource conditions . |
| Approach: | They propose two methods to incorporate phone features into current neural speech translation models. |
| Outcome: | The proposed models outperform existing models and cascades by up to 9 BLEU on low-resource conditions. |
Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASR (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous speech-to-text translation suffer from error propagation and extra latency. |
| Approach: | They propose a new paradigm for simultaneous speech-to-text translation using two separate decoders . they use multitask learning to jointly learn these two tasks with a shared encoder . |
| Outcome: | The proposed method achieves substantially better translation quality at similar levels of latency. |
Learning When to Translate for Streaming Speech (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods waiting-and-translating for a fixed duration break speech acoustic units . Existing models waiting-for a set duration and generating partial sentences are not effective . |
| Approach: | They propose a monotonic segmentation module inside an encoder-decoder model to detect proper speech unit boundaries for a streaming speech input. |
| Outcome: | The proposed method outperforms existing methods on a speech translation dataset and achieves the best trade-off between translation quality and latency. |
Segmenting Subtitles for Correcting ASR Segmentation Errors (2021.eacl-main)
Copied to clipboard
David Wan, Chris Kedzie, Faisal Ladhak, Elsbeth Turcan, Petra Galuscakova, Elena Zotkina, Zhengping Jiang, Peter Bell, Kathleen McKeown
| Challenge: | Typical ASR systems segment input audio into utterances using purely acoustic information, which may not resemble sentence-like units expected by conventional machine translation systems for spoken language translation (SLT). |
| Approach: | They propose a model for correcting ASR acoustic segmentation using subtitles as a proxy dataset for creating synthetic aural utterances by modeling common error modes. |
| Outcome: | The proposed model improves performance on MT and audio-document cross-language information retrieval (CLIR) it uses subtitles as a proxy dataset to correct ASR acoustic segmentation . |
Tutorial: End-to-End Speech Translation (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | Speech translation is the translation of speech in one language typically to text in another, traditionally accomplished through a combination of automatic speech recognition and machine translation. |
| Approach: | This tutorial introduces the techniques used in cutting-edge research on speech translation. |
| Outcome: | The proposed models achieve state-of-the-art performance with end-to-end speech translation for both high- and low-resource languages. |
Thesis Proposal: Development of End-to-End Speech Translation Models for Indian Languages (2026.eacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to speech-to-speech translation rely on cascaded pipelines . current approaches rely only on text representations, but they suffer from errors and latency . a new direct speech translation framework is proposed to bridge linguistic gaps . |
| Approach: | They propose a sequence-to-sequence direct speech translation framework that can translate speech from one Indian language to another without relying on intermediate text representations. |
| Outcome: | The proposed framework can translate speech from one Indian language to another without relying on intermediate text representations. |