Cross Attention Augmented Transducer Networks for Simultaneous Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous translation are limited by monotonic constraint . a novel architecture for simultaneous translation is proposed . |
| Approach: | They propose a cross attention-augmented transducer for simultaneous translation that optimizes both policies and translation models by expanding target sequences with blank symbols. |
| Outcome: | The proposed architecture achieves better latency-quality trade-offs than state-of-the-art approaches. |
Similar Papers
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks (2023.acl-long)
Copied to clipboard
| Challenge: | Neural based end-to-end frameworks have achieved remarkable success in speech-totext tasks, such as automatic speech recognition (ASR) and speech- totext translation (ST). |
| Approach: | They propose to combine Transducer and Attention based Encoder-Decoder (TAED) for speech-to-text tasks and leverage AED's strength in non-monotonic sequence to sequence learning while retaining Transducers streaming property. |
| Outcome: | The proposed model outperforms Transducer and Attention based Encoder-Decoder (TAED) on the MuST-C dataset and shows that it is not bound by any specific language model. |
Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing models for automatic speech recognition and multilingual speech translation are on par with cascade counterparts. |
| Approach: | They propose a dual-decoder Transformer architecture that performs automatic speech recognition and multilingual speech translation. |
| Outcome: | The proposed models outperform the previously-reported highest translation performance in multilingual settings and bilingual one-to-one results. |
Attention as a Guide for Simultaneous Speech Translation (2023.acl-long)
Copied to clipboard
| Challenge: | EDAtt uses attention patterns to determine when to emit partial translations . results show that it yields better results compared to existing SimulST policies . |
| Approach: | They propose an adaptive policy that exploits attention patterns between audio source and target textual translation to guide an offline-trained ST model during simultaneous inference. |
| Outcome: | The proposed policy yields better results on en->de, compared to the current state of the art. |
Language Model Augmented Monotonic Attention for Simultaneous Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing adaptive policies for simultaneous neural machine translation use monotonic attention to perform read/write decisions based on the partial source and target sequences. |
| Approach: | They propose a framework to aid monotonic attention with an external language model to improve its decisions. |
| Outcome: | The proposed approach improves on English-German and English-French translation tasks by using a language model. |
AdaST: Dynamically Adapting Encoder States in the Decoder for End-to-End Speech-to-Text Translation (2021.findings-acl)
Copied to clipboard
| Challenge: | End-to-end speech translation models learn acoustic representations from the encoder, which is not desirable for cross-modal and cross-lingual translation. |
| Approach: | They propose an adaptive speech-to-text translation model that dynamically adapts acoustic states in the decoder. |
| Outcome: | The proposed model outperforms state-of-the-art speech translation models on two widely-used datasets. |
Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | a series of experiments show that fine-tuning only the cross-attention parameters is nearly as effective as fine-timing all parameters. |
| Approach: | They conduct experiments to fine-tune a translation model on data where either the source or target language has changed. |
| Outcome: | The proposed model can be trained to several new languages with reduced parameter storage overhead. |
LCMA-SRT: Language-Conditional Mixture-of-Experts Adapters for Joint Multilingual Speech Recognition and Translation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing hierarchical transducers suffer from negative transfer and unstable target-language generation, while training separate models for each direction is computationally prohibitive. |
| Approach: | They propose a hierarchical transducer with language-conditional Mixture-of-Experts adapters to improve multilingual joint automatic speech recognition and speech translation. |
| Outcome: | Experiments on Europarl-ST (9 languages, 72 directions) show that LCMA-SRT improves both ASR and ST within a single joint model, reducing average WER and improving BLEU and COMET over strong hierarchical transducer baselines. |
DrFrattn: Directly Learn Adaptive Policy from Attention for Simultaneous Machine Translation (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to learn read/write policies from attention mechanism may compromise effectiveness of attention mechanism . |
| Approach: | They propose a method that directly learns adaptive policies from the attention mechanism . experimental results demonstrate that the method achieves an improved balance between translation accuracy and latency. |
| Outcome: | The proposed method achieves improved balance between translation accuracy and latency. |
Duplex Diffusion Models Improve Speech-to-Speech Translation (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to speech-to-speech translation train two separate models or a multitask-learned model with low efficiency and inferior performance. |
| Approach: | They propose a duplex diffusion model that applies diffusion probabilistic models to both sides of a reversible duplex Conformer and enables reverse speech translation by simply flipping the input and output ends. |
| Outcome: | The proposed model achieves the first success of reversible speech translation with significant improvements of ASR-BLEU scores compared with a list of state-of-the-art baselines. |
Decoder-only Streaming Transformer for Simultaneous Translation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods for siMT focus on the Encoder-Decoder architecture, but there are limitations in training and inference. |
| Approach: | They propose a model that generates translation while reading source tokens . they propose Streaming Self-Attention mechanism tailored for the Decoder-only architecture . |
| Outcome: | The proposed model achieves state-of-the-art performance on three translation tasks. |