Papers by Mireia Farrús
TEMA: Token Embeddings Mapping for Enriching Low-Resource Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Low-resource languages, that is, languages that do not have a massive amount of text, risk being almost excluded from the possibility of having good NLP applications. |
| Approach: | They propose an algorithm that maps token embeddings of a richly pre-trained model to a poorly trained model and creates a more complex model. |
| Outcome: | The proposed model reduces perplexity and is competitive or better for the most semantic tasks. |
Recycle Your Wav2Vec2 Codebook: A Speech Perceiver for Keyword Spotting (2022.coling-1)
Copied to clipboard
| Challenge: | Pretraining a keyword Spotting model with a pretraining encoder is expensive and requires a quadratic cost. |
| Approach: | They propose to recycle phonetic information encoded in wav2vec2.0's latent codebook, which has been typically thrown away after pretraining. |
| Outcome: | The proposed model can be initialized with phonetic embeddings, and it delivers accuracy gains at no latency costs. |
Frequency Balanced Datasets Lead to Better Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evidence that high-frequency tokens in pretraining data might bias learning, causing undesired effects, is not clear. |
| Approach: | They propose a sampling algorithm that iteratively assesses token frequencies and removes sentences that contain still high-frequency tokens, resulting in a balanced dataset. |
| Outcome: | The proposed method reduces the amount of pre-training data required for training attention-based transformer language models by up to three times. |
Compilation of Corpora for the Study of the Information Structure–Prosody Interface (L18-1)
Copied to clipboard
| Challenge: | empirical studies on the Information Structure-prosody interface are scarce . thematicity defines how content is packaged in terms of "what is being talked about" a different view on thematicality is advocated by I. Mel'uk in the context of the MTT. |
| Approach: | They propose a method for the compilation of annotated corpora to study the correspondence between Information Structure and prosody. |
| Outcome: | The proposed method is applied to a corpus of read speech in English annotated with hierarchical thematicity and automatically extracted prosodic parameters. |