SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation (2020.aacl-main)
Copied to clipboard
| Challenge: | Using end-to-end Simultaneous text translation, we adapt wait-k and monotonic multihead attention to end- to-end simultaneous speech translation. |
| Approach: | They propose to combine a fixed and flexible pre-decision module with fixed and flexibility policies to adapt simultaneous text translation methods such as wait-k and monotonic multihead attention to end-to-end simultaneous speech translation. |
| Outcome: | The proposed method can generate translations with maximum quality and minimal latency, targeting video caption translations and real-time language interpreter. |
Similar Papers
Training Simultaneous Speech Translation with Robust and Random Wait-k-Tokens Strategy (2023.emnlp-main)
Copied to clipboard
| Challenge: | Simultaneous Speech Translation (SimulST) is a task focused on ensuring high-quality translation of speech in low-latency situations. |
| Approach: | They propose a token-level cross-modal alignment method to improve the translation of text to audio . they use audio transcription pairs to pre-train the encoder and a random wait-k-tokens strategy to optimize the task. |
| Outcome: | The proposed method achieves better trade-off between translation quality and latency. |
CA*: Addressing Evaluation Pitfalls in Computation-Aware Latency for Simultaneous Speech Translation (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing metrics for Simultaneous speech translation (SimulST) are inaccurately measuring latency in unsegmented streaming settings. |
| Approach: | They propose to modify existing metrics to correctly measure computation-aware latency for SimulST systems, addressing limitations present in existing metrics. |
| Outcome: | The proposed model is based on a real-time, lowlatency scenario where the model starts generating the textual translation before the entire audio input is processed. |
SimulSpeech: End-to-End Simultaneous Speech to Text Translation (2020.acl-main)
Copied to clipboard
| Challenge: | SimulSpeech is an end-to-end simultaneous speech to text translation system . conventional approaches to simultaneous speech translation divide the translation process into two stages . |
| Approach: | They develop an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently. |
| Outcome: | The proposed system achieves reasonable BLEU scores and lower delay compared to full-sentence translation model. |
Going Beyond Your Expectations in Latency Metrics for Simultaneous Speech Translation (2025.findings-acl)
Copied to clipboard
| Challenge: | Current evaluation practices in Simultaneous Speech Translation systems involve segmenting the input audio and its translations, calculating quality and latency metrics for each segment, and averaging the results. |
| Approach: | They propose to use the mean to estimate latency for Simultaneous Speech Translation systems to provide a better understanding of their results. |
| Outcome: | The proposed methods can provide a better understanding of SimulST systems’ latency. |
Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to perform adaptive and fixed translations lack evaluation before taking actions. |
| Approach: | They propose a method to perform adaptive translation policy via post-evaluation into fixed policy . their method evaluates rationality of next action by measuring change in source content . |
| Outcome: | The proposed method exceeds strong baselines under all latency. |
Does Simultaneous Speech Translation need Simultaneous Models? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Simultaneous speech translation (SimulST) systems strive for high output quality but also low latency. |
| Approach: | They propose to train SimulST offline without additional training or adaptation . they also show offline training achieves similar or better quality compared to offline training . |
| Outcome: | The proposed model can serve both offline and simultaneous applications without additional training or adaptation. |
Adaptive Policy with Wait-k Model for Simultaneous Translation (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to simultaneous machine translation require a robust read/write policy . a standalone multi-path wait-k model performs competitively with adaptive policies . |
| Approach: | They propose a more flexible approach by decoupling the adaptive policy model from the translation model. |
| Outcome: | The proposed approach outperforms baseline approaches in translation tasks. |
Fluent and Low-latency Simultaneous Speech-to-Speech Translation with Self-adaptive Training (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Current approaches to simultaneous speech-to-speech translation accumulate more and more latencies in later sentences when the speaker talks faster. |
| Approach: | They propose a method which generates more fluent target speech latency than the baseline . they propose to use self-adaptive translation to adjust the length of translations to accommodate different source speech rates. |
| Outcome: | Xiong et al., 2019) show that the proposed method generates more fluent target speech latency than baseline . authors say it provides more natural communication process than speech-to-text translation . xiong and colleagues say the proposed technique is more efficient than current approaches . |
SIMULEVAL: An Evaluation Toolkit for Simultaneous Translation (2020.emnlp-demos)
Copied to clipboard
| Challenge: | SimulEval is an evaluation toolkit for simultaneous text and speech translation. |
| Approach: | They propose a server-client scheme for simultaneous translation that uses server input and client policies to evaluate models. |
| Outcome: | The proposed evaluation toolkit is available for both text and speech translation. |
Learning Adaptive Segmentation Policy for End-to-End Simultaneous Translation (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods to perform simultaneous speech-to-text translation ignore contextual information and suffer from low translation quality. |
| Approach: | They propose an adaptive segmentation policy for simultaneous speech-to-text translation . it learns to segment the source streaming speech into meaningful units . |
| Outcome: | The proposed method achieves a good accuracy-latency trade-off over state-of-the-art methods on English-German and Chinese-English. |