Papers by Taido Purason
To Err Is Human, but Llamas Can Learn It Too (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Specifically, we fine-tune Llama 2 LMs for error generation and find that this approach yields synthetic errors akin to human errors. |
| Approach: | They propose to fine-tune Llama 2 LMs for error generation and train GEC Llma models using these artificial errors. |
| Outcome: | The proposed approach outperforms state-of-the-art models with gains ranging between 0.8 and 6 F0.5 points across all languages tested. |
Teaching Old Tokenizers New Words: Efficient Tokenizer Adaptation for Pretrained Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Extending existing vocabulary is a widely used step in adapting pre-trained language models to new domains or languages. |
| Approach: | They propose to extend a pre-trained tokenizer by continuing the BPE merge learning process on new data. |
| Outcome: | The proposed method improves tokenization efficiency and improves model utilization. |
Teaching Llama a New Language Through Cross-Lingual Knowledge Transfer (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing methods to adapt pretrained Large Language Models to new lower-resource languages are limited to English. |
| Approach: | They propose to combine cross-lingual instruction-tuning with additional monolingual pretraining to adapt LLMs to new lower-resource languages. |
| Outcome: | The proposed model is the first open-source instruction-following LLM for Estonian . the proposed model improves commonsense reasoning and multi-turn conversation capabilities . |
LLMs for Extremely Low-Resource Finno-Ugric Languages (2025.findings-naacl)
Copied to clipboard
| Challenge: | Low-resource languages such as those in the Finno-Ugric family are underrepresented in large language models. |
| Approach: | They propose to develop large language models for extremely low-resource languages . they focus on Vro, Livonian, and Komi, which are underrepresented . |
| Outcome: | The proposed models cover almost the entire cycle of creation, from data collection to instruction tuning and evaluation. |
Multilinguality or Back-translation? A Case Study with Estonian (2024.lrec-main)
Copied to clipboard
| Challenge: | a limited amount of parallel data is available for machine translation, and synthetic data is often used to improve translation quality. |
| Approach: | They propose a large-scale synthetic corpus of Estonian translations that contains over 1 billion parallel sentences. |
| Outcome: | The proposed model improves the baseline model while maintaining multilinguality . the proposed model is 6 times larger than the Estonian corpus and twice the size of the Estonial part of the CulturaX corpus. |