Papers by Marta Costa-jussà
On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Several algorithms implemented by language models have been successfully reverse-engineered, leaving it unclear how universal circuits are across different settings. |
| Approach: | They propose to use Gemma 2B to solve the subject-verb agreement task across two different languages, English and Spanish. |
| Outcome: | The proposed circuits solve the subject-verb agreement task across two different languages, and are language-independent and language-dependent. |
SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations (2023.findings-emnlp)
Copied to clipboard
| Challenge: | End-to-end Speech Translation models are limited by a data bottleneck . end-to end models can address several shortcomings of cascaded models . |
| Approach: | They propose a data augmentation strategy to augment sentence-level datasets by using an Audio Segmentation system to re-segment the speech of each document with different length constraints. |
| Outcome: | The proposed method achieves state-of-the-art results in MuST-C and in mTEDx. |
Unveiling the Role of Pretraining in Direct Speech Translation (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to train direct speech-to-text translation systems are pretraining the encoder on automatic speech recognition, thus losing efficiency in the training process. |
| Approach: | They propose to change the decoder cross-attention to integrate source information from earlier steps in training. |
| Outcome: | The proposed model can achieve comparable performance to the pretrained model while reducing training time. |
Pushing the Limits of Zero-shot End-to-End Speech Translation (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing approaches to end-to-end Speech Translation (ST) systems require limited data, which can cause data scarcity and performance degradation. |
| Approach: | They propose a method for zero-shot ST that bridges the modality gap without any paired ST data. |
| Outcome: | The proposed method bridges the modality gap without any paired ST data on a speech encoder and on MT models. |
Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scale (2023.emnlp-main)
Copied to clipboard
Marta Costa-jussà, Pierre Andrews, Eric Smith, Prangthip Hansanti, Christophe Ropers, Elahe Kalbassi, Cynthia Gao, Daniel Licht, Carleigh Wood
| Challenge: | Multilingual HolisticBias dataset includes 20,459 sentences in 50 languages . dataset is intended to uncover demographic imbalances and quantify mitigations . |
| Approach: | They propose a multilingual extension of the HolisticBias dataset . they use 118 demographic descriptors and three patterns to build multilingual sentences . |
| Outcome: | The proposed model improves translation quality when the source input only differs in gender . it also improves when the masculine human reference is used in the model . |
BLASER 2.0: a metric for evaluation and quality estimation of massively multilingual speech and text translation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Automatic evaluation of machine translation (MT) is difficult because of the number of possible ways to express a thought in a language. |
| Approach: | They propose to use BLASER 2.0 to evaluate machine translation quality . they propose to apply the reference-based model to a sentence-based version . |
| Outcome: | The proposed model is applicable to detecting translation hallucinations and filtering training datasets to obtain more reliable translation models. |
MuTox: Universal MUltilingual Audio-based TOXicity Dataset and Zero-shot Detector (2024.findings-acl)
Copied to clipboard
Marta Costa-jussà, Mariano Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alexandre Mourachko, Christophe Ropers, Carleigh Wood
| Challenge: | Existing studies on text-based toxicity detection for other languages are limited, especially for languages other than English. |
| Approach: | They propose a multilingual audio-based toxicity classifier which covers 14 different linguistic families and a dataset of 20,000 audio utterances for English and Spanish. |
| Outcome: | The new classifier improves F1-Score by an average of 100% when compared to existing wordlist-based classifiers. |
Toxicity in Multilingual Machine Translation at Scale (2023.findings-emnlp)
Copied to clipboard
Marta Costa-jussà, Eric Smith, Christophe Ropers, Daniel Licht, Jean Maillard, Javier Ferrando, Carlos Escolano
| Challenge: | In this paper, we evaluate and analyze added toxicity when translating a large dataset from English into 164 languages. |
| Approach: | They evaluate added toxicity when translating a large dataset from English into 164 languages. |
| Outcome: | The results show that added toxicity is more prevalent in low-resource languages than in high-resolution translations. |
HalOmi: A Manually Annotated Benchmark for Multilingual Hallucination and Omission Detection in Machine Translation (2023.emnlp-main)
Copied to clipboard
David Dale, Elena Voita, Janice Lam, Prangthip Hansanti, Christophe Ropers, Elahe Kalbassi, Cynthia Gao, Loic Barrault, Marta Costa-jussà
| Challenge: | Previously available quality assessments do not distinguish between hallucinations and omissions. |
| Approach: | They propose to annotate hallucinations and omissions in machine translation using a single language pair. |
| Outcome: | The proposed dataset covers 18 translation directions with varying resource levels and scripts. |