Papers by Rodolfo Zevallos
TEMA: Token Embeddings Mapping for Enriching Low-Resource Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Low-resource languages, that is, languages that do not have a massive amount of text, risk being almost excluded from the possibility of having good NLP applications. |
| Approach: | They propose an algorithm that maps token embeddings of a richly pre-trained model to a poorly trained model and creates a more complex model. |
| Outcome: | The proposed model reduces perplexity and is competitive or better for the most semantic tasks. |
Hints on the data for language modeling of synthetic languages with transformers (2023.acl-long)
Copied to clipboard
| Challenge: | Language Models (LMs) are becoming more useful for providing representations for NLP applications. |
| Approach: | They evaluated whether the critical amount of data varies for different morphological typologies . they found that the size of the vocabulary due to morphology is directly correlated with LM perplexity . |
| Outcome: | The proposed method reduces perplexity by more than a half for a polysynthetic language like Quechua . |
Frequency Balanced Datasets Lead to Better Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing evidence that high-frequency tokens in pretraining data might bias learning, causing undesired effects, is not clear. |
| Approach: | They propose a sampling algorithm that iteratively assesses token frequencies and removes sentences that contain still high-frequency tokens, resulting in a balanced dataset. |
| Outcome: | The proposed method reduces the amount of pre-training data required for training attention-based transformer language models by up to three times. |
WordNet-QU: Development of a Lexical Database for Quechua Varieties (2022.coling-1)
Copied to clipboard
| Challenge: | Quechua is a low-resource language from south America but lacks resources to build high-performance computational systems. |
| Approach: | They propose to include Quechua in a lexical database called wordnet . they propose a synset alignment algorithm to compare Quechuan to its nearest high-resource language . |
| Outcome: | The proposed system compares Quechua to its nearest high-resource language, Spanish . it uses a synset alignment algorithm to find Quechuan resources in a lexical database . |
Huqariq: A Multilingual Speech Corpus of Native Languages of Peru forSpeech Recognition (2022.lrec-1)
Copied to clipboard
| Challenge: | the Huqariq corpus is a multilingual collection of speech from native Peruvian languages . the project is designed to preserve endangered languages in the public domain . |
| Approach: | They propose to use crowdsourcing to collect transcribed audio from native Peruvian languages . they propose to do 220 hours of speech recognition experiments to verify quality . |
| Outcome: | The Huqariq corpus is a multilingual collection of speech from native Peruvian languages . the project is expected to reach 20 native languages out of 48 native languages by 2022 . |