| Challenge: | Recent advances in speech-to-text translation have led to significant improvements, but the availability of appropriate training data is limiting. |
| Approach: | They propose a new text-to-speech and speech-tospech translation model that directly learns to generate the speech signal based on the pronunciation of the source language. |
| Outcome: | The proposed model learns to generate speech signal based on pronunciation of source language. |
Similar Papers
LibriVoxDeEn: A Corpus for German-to-English Speech Translation and German Speech Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | a corpus of sentence-aligned triples of German audio, German text, and English translation is available for speech recognition . a large corpus is available to date for end-to-end speech translation based on parallel data . |
| Approach: | They present a corpus of sentence-aligned triples of German audio, German text, and English translation based on German audio books. |
| Outcome: | The proposed corpus is the largest resource for German speech recognition and for end-to-end German-to English speech translation. |
CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing datasets involve language pairs with English as source language, are low resource or lack labeled data. |
| Approach: | They propose a multilingual speech-to-text translation corpus from 11 languages into English . they provide empirical evidence of the quality of the data and provide initial benchmarks . |
| Outcome: | The proposed model is the first end-to-end multilingual model for spoken language translation. |
SDS-200: A Swiss German Speech to Standard German Text Corpus (2022.lrec-1)
Copied to clipboard
Michel Plüss, Manuela Hürlimann, Marc Cuny, Alla Stöckli, Nikolaos Kapotis, Julia Hartmann, Malgorzata Anna Ulasik, Christian Scheller, Yanick Schraner, Amit Jain, Jan Deriu, Mark Cieliebak, Manfred Vogel
| Challenge: | Using a web recording tool, participants were asked to translate their Swiss German text to their own dialect before recording it. |
| Approach: | They present a corpus of Swiss German dialectal speech with Standard German text translations . the dataset allows for training speech translation, dialect recognition, and speech synthesis systems . |
| Outcome: | The dataset allows for training speech translation, dialect recognition, and speech synthesis systems. |
Speech-to-Speech Translation for a Real-world Unwritten Language (2023.findings-acl)
Copied to clipboard
Peng-Jen Chen, Kevin Tran, Yilin Yang, Jingfei Du, Justine Kao, Yu-An Chung, Paden Tomasello, Paul-Ambroise Duquenne, Holger Schwenk, Hongyu Gong, Hirofumi Inaguma, Sravya Popuri, Changhan Wang, Juan Pino, Wei-Ning Hsu, Ann Lee
| Challenge: | a new study examines speech-to-speech translation (S2ST) that translates speech from one language into another . the research area for unwritten languages remains a research area with little exploration due to the lack of training data. |
| Approach: | They propose a system that translates speech from one language into another . they use Taiwanese Hokkien as an example of an unwritten language . |
| Outcome: | The proposed system can be used to train models in languages without standard writing systems. |
Augmenting Librispeech with French Translations: A Multimodal Corpus for Direct Speech Translation Evaluation (L18-1)
Copied to clipboard
| Challenge: | Recent work in spoken language translation (SLT) has attempted to build end-to-end speech-totext translation without using source language transcription during learning or decoding. |
| Approach: | They propose to augment an existing (monolingual) corpus: LibriSpeech. |
| Outcome: | The proposed corpus is derived from read audiobooks from the LibriVox project and has been carefully segmented and aligned. |
Tutorial: End-to-End Speech Translation (2021.eacl-tutorials)
Copied to clipboard
| Challenge: | Speech translation is the translation of speech in one language typically to text in another, traditionally accomplished through a combination of automatic speech recognition and machine translation. |
| Approach: | This tutorial introduces the techniques used in cutting-edge research on speech translation. |
| Outcome: | The proposed models achieve state-of-the-art performance with end-to-end speech translation for both high- and low-resource languages. |
STT4SG-350: A Speech Corpus for All Swiss German Dialect Regions (2023.acl-short)
Copied to clipboard
Michel Plüss, Jan Deriu, Yanick Schraner, Claudio Paonessa, Julia Hartmann, Larissa Schmidt, Christian Scheller, Manuela Hürlimann, Tanja Samardžić, Manfred Vogel, Mark Cieliebak
| Challenge: | We present a corpus of Swiss German speech annotated with Standard German text at the sentence level. |
| Approach: | They present a corpus of Swiss German speech annotated with Standard German sentences . they use a web app to show the speakers standard German sentences and record them . |
| Outcome: | The corpus contains 343 hours of speech from all Swiss German dialect regions . it is the largest public speech corpus for Swiss German to date . |
Speech Translation and the End-to-End Promise: Taking Stock of Where We Are (2020.acl-main)
Copied to clipboard
| Challenge: | Until recently, the only feasible approach to translating acoustic speech signals into text was the cascaded approach. |
| Approach: | They propose a classification of the main challenges of traditional approaches to speech translation . they argue that end-to-end models fall short due to compromises made to address data scarcity . |
| Outcome: | This paper provides a brief survey of the main challenges of traditional approaches in speech translation . it reveals that many end-to-end models fail due to compromises made to address data scarcity. |
KazakhTTS2: Extending the Open-Source Kazakh TTS Corpus With More Data, Speakers, and Topics (2022.lrec-1)
Copied to clipboard
| Challenge: | Text-to-speech (TTS) is a process of converting written text into speech. |
| Approach: | They present an expanded version of their text-to-speech corpus for Kazakh . they propose to use the corpus to build high-quality TTS systems for the language . |
| Outcome: | The constructed corpus is sufficient to build robust TTS models for Kazakh and other Turkic languages, with a subjective mean opinion score ranging from 3.6 to 4.2 for all the five speakers. |
The WAW Corpus: The First Corpus of Interpreted Speeches and their Translations for English and Arabic (L18-1)
Copied to clipboard
| Challenge: | Using the corpus, we study the characteristics of interpreters' work and train machine translation systems. |
| Approach: | They propose to build an interpreting corpus for Arabic and an Arabic corpus to study interpreters' work. |
| Outcome: | The proposed corpus can be used for teaching interpreters and to train machine translation systems. |