Papers by Sara Papi
Does Simultaneous Speech Translation need Simultaneous Models? (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Simultaneous speech translation (SimulST) systems strive for high output quality but also low latency. |
| Approach: | They propose to train SimulST offline without additional training or adaptation . they also show offline training achieves similar or better quality compared to offline training . |
| Outcome: | The proposed model can serve both offline and simultaneous applications without additional training or adaptation. |
Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora (2022.aacl-short)
Copied to clipboard
| Challenge: | Existing models for subtitling require parallel data paired with audio inputs and textual translations. |
| Approach: | They propose to convert existing ST corpora into SubST resources without human intervention by exploiting audio and text in a multimodal fashion. |
| Outcome: | The proposed model achieves high segmentation quality in zero-shot conditions with manual and automatic segmentation. |
What the Harm? Quantifying the Tangible Impact of Gender Bias in Machine Translation with a Human-centered Study (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing bias measurements do not reflect the gender disparities found in machine translation. |
| Approach: | They conduct a human-centered study to examine if and to what extent bias in machine translation brings harms with tangible costs, such as quality of service gaps between women and men. |
| Outcome: | The findings advocate for human-centered approaches that can inform the societal impact of bias. |
Direct Speech Translation for Automatic Subtitling (2023.tacl-1)
Copied to clipboard
| Challenge: | Existing models for automatic subtitling generate subtitles in the target language along with their timestamps. |
| Approach: | They propose a direct speech translation model that generates subtitles in the target language along with their timestamps with a single model. |
| Outcome: | The proposed model outperforms a cascade system on 7 language pairs and on new benchmarks. |
Speechformer: Reducing Information Loss in Direct Speech Translation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Current approaches to speech-to-text translation (ST) use a pipeline of two sub-components - an automatic speech recognition (ASR) and a machine translation (MT) model. |
| Approach: | They propose an architecture that avoids initial lossy compression and aggregates information only at a higher level according to more informed linguistic criteria. |
| Outcome: | The proposed architecture achieves gains of up to 0.8 BLEU on the standard MuST-C corpus and up to 4.0 BLUE in a low resource scenario. |
StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection (2024.acl-long)
Copied to clipboard
| Challenge: | Existing studies on streaming translation focus on SimulST only focusing on StreamST . StreamAtt is the first Stream ST policy and proposes StreamLAAL . |
| Approach: | They propose StreamAtt, the first StreamST policy, and StreamLAAL, the second Stream ST latency metric. |
| Outcome: | Experiments in 8 languages show that StreamAtt is more efficient than SimulST . StreamLAAL is the first StreamST latency metric comparable with existing metrics for Simul ST. |
SBAAM! Eliminating Transcript Dependency in Automatic Subtitling (2024.acl-long)
Copied to clipboard
| Challenge: | Subtitling is a crucial task for enhancing the accessibility of audiovisual content and relying on automatic transcripts for the three subtasks is uncharted territory. |
| Approach: | They propose a model capable of producing automatic subtitles, completely eliminating any dependence on intermediate transcripts also for timestamp prediction. |
| Outcome: | Experimental results show that the proposed model eliminates the need for intermediate transcripts for timestamp prediction across multiple language pairs and diverse conditions. |
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have been successful in NLP tasks, but there is growing interest in extending their capabilities to speech. |
| Approach: | They propose to use dense feature prepending (DFP) to integrate speech into LLMs to enable end-to-end training with a speech encoder. |
| Outcome: | The proposed approach does not show a clear advantage over cross-attention. |
Integrating Language Models into Direct Speech Translation: An Inference-Time Solution to Control Gender Inflection (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing solutions to control speaker-related gender inflections in ST involve dedicated model retraining on gender-labeled data. |
| Approach: | They propose to use a gender-based inference-time solution to control speaker-related gender inflections in ST by replacing the implicitly learned internal language model with gender-specific external LMs. |
| Outcome: | The proposed approach outperforms the base models and the best training-time mitigation strategy by up to 31.0 and 1.6 points in gender accuracy, respectively, for feminine forms. |
When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP (2024.acl-long)
Copied to clipboard
| Challenge: | despite its crucial role in research experiments, code correctness is often presumed on the perceived quality of results. |
| Approach: | They propose to promote code-quality checklists to promote coding best practices . they propose to fix bugs in conformer implementations to mitigate this risk . |
| Outcome: | The proposed checklists aim to promote coding best practices and improve software quality within the NLP community. |
MOSEL: 950,000 Hours of Speech Data for Open-Source Speech Foundation Model Training on EU Languages (2024.emnlp-main)
Copied to clipboard
Marco Gaido, Sara Papi, Luisa Bentivogli, Alessio Brutti, Mauro Cettolo, Roberto Gretter, Marco Matassoni, Mohamed Nabih, Matteo Negri
| Challenge: | Existing speech FMs fall short of full compliance with open-source principles . existing models do not have model weights, code, and training data publicly available . |
| Approach: | They propose to use a CC-BY license to create open-source speech FMs for EU languages . they collect suitable training data by surveying automatic speech recognition datasets . |
| Outcome: | The proposed model can be used in the 24 official languages of the European Union. |
Attention as a Guide for Simultaneous Speech Translation (2023.acl-long)
Copied to clipboard
| Challenge: | EDAtt uses attention patterns to determine when to emit partial translations . results show that it yields better results compared to existing SimulST policies . |
| Approach: | They propose an adaptive policy that exploits attention patterns between audio source and target textual translation to guide an offline-trained ST model during simultaneous inference. |
| Outcome: | The proposed policy yields better results on en->de, compared to the current state of the art. |
How Do Hyenas Deal with Human Speech? Speech Recognition and Translation with ConfHyena (2024.lrec-main)
Copied to clipboard
| Challenge: | Currently, attention-based models face computational hurdles in processing long sequences due to its quadratic complexity. |
| Approach: | They propose a conformer whose encoder self-attentions are replaced with Hyena for speech processing . they propose 'confhyena' model that reduces training time by 27% at minimal cost . |
| Outcome: | The proposed model reduces training time by 27% at the cost of minimal quality degradation. |