Papers by Martijn Bartelds
Adapting Monolingual Models: Data can be Scarce when Language Similarity is High (2021.findings-acl)
Copied to clipboard
| Challenge: | Large pre-trained language models are the dominant approach for solving many tasks in natural language processing. |
| Approach: | They propose to retrain the lexical layers of four BERT-based models using data from two low-resource target languages while the Transformer layers are independently finetuned on a POS-tagging task in the model's source language. |
| Outcome: | The proposed method achieves high performance for both target and target languages with high similarity. |
Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation (2023.acl-long)
Copied to clipboard
| Challenge: | Using self-training or text-to-speech (TTS) to improve low-resource ASR performance is costly and can lead to catastrophic forgetting. |
| Approach: | They examine whether data augmentation techniques could help improve low-resource ASR performance . they use self-training to generate transcriptions, which are combined with original data to train new system . |
| Outcome: | The proposed approach yields a 20.5% reduction in WER compared to a system trained on 24 minutes of manually transcribed speech. |
False Friends Are Not Foes: Investigating Vocabulary Overlap in Multilingual Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Prior work has shown that token overlap facilitates cross-lingual transfer or introduces interference between languages? |
| Approach: | They devised a controlled experiment where they train bilingual autoregressive models on multiple language pairs under systematically varied vocabulary overlap settings. |
| Outcome: | The proposed model outperforms models with disjointed vocabularies on XNLI and XQuAD and shows that token overlap is beneficial for multilingual tokenizers. |
Quantifying Language Variation Acoustically with Few Resources (2022.naacl-main)
Copied to clipboard
| Challenge: | acoustic models represent linguistic information based on massive amounts of data. |
| Approach: | They examine the model's ability to distinguish low-resource (Dutch) regional varieties by extracting embeddings from hidden layers and dynamic time warping. |
| Outcome: | The proposed model outperforms transcription-based models without phonetic transcriptions on the basis of only six seconds of speech. |