Papers by Gerard Gállego
Unveiling the Role of Pretraining in Direct Speech Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train direct speech-to-text translation systems are pretraining the encoder on automatic speech recognition, thus losing efficiency in the training process. |
| Approach: | They propose to change the decoder cross-attention to integrate source information from earlier steps in training. |
| Outcome: | The proposed model can achieve comparable performance to the pretrained model while reducing training time. |
Pushing the Limits of Zero-shot End-to-End Speech Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to end-to-end Speech Translation (ST) systems require limited data, which can cause data scarcity and performance degradation. |
| Approach: | They propose a method for zero-shot ST that bridges the modality gap without any paired ST data. |
| Outcome: | The proposed method bridges the modality gap without any paired ST data on a speech encoder and on MT models. |
Multiformer: A Head-Configurable Transformer-Based Model for Direct Speech Translation (2022.naacl-srw)
Copied to clipboard
| Challenge: | Existing approaches to address speech tasks with a self-attention mechanism are expensive and lead to information loss. |
| Approach: | They propose a Transformer-based model which uses different attention mechanisms on each head to bias the self-attention towards the extraction of more diverse token interactions. |
| Outcome: | The proposed model outperforms baseline models by 0.7 BLEU in the speech task. |