Papers by Veronika Laippala
Aurora-M: Open Source Continual Pre-training for Multilingual Language and Code (2025.coling-industry)
Copied to clipboard
Taishi Nakamura, Mayank Mishra, Simone Tedeschi, Yekun Chai, Jason T. Stillerman, Felix Friedrich, Prateek Yadav, Tanmay Laud, Vu Minh Chien, Terry Yue Zhuo, Diganta Misra, Ben Bogin, Xuan-Son Vu, Marzena Karpinska, Arnav Varma Dantuluri, Wojciech Kusa, Tommaso Furlanello, Rio Yokota, Niklas Muennighoff, Suhas Pai, Tosin Adewumi, Veronika Laippala, Xiaozhe Yao, Adalberto Barbosa Junior, Aleksandr Drozd, Jordan Clive, Kshitij Gupta, Liangyu Chen, Qi Sun, Ken Tsui, Nour Moustafa-Fahmy, Nicolo Monti, Tai Dang, Ziyang Luo, Tien-Tung Bui, Roberto Navigli, Virendra Mehta, Matthew Blumberg, Victor May, Hiep Nguyen, Sampo Pyysalo
| Challenge: | Pretrained language models are integral part of AI applications, but their high computational cost limits accessibility. |
| Approach: | They evaluate Aurora-M, a 15B parameter multilingual open-source model trained on English, Finnish, Hindi, Japanese, Vietnamese, and code. |
| Outcome: | The proposed model outperforms existing models on English, Finnish, Hindi, Japanese, Vietnamese, and code. |
Beyond the English Web: Zero-Shot Cross-Lingual and Lightweight Monolingual Classification of Registers (2021.eacl-srw)
Copied to clipboard
Liina Repo, Valtteri Skantsi, Samuel Rönnqvist, Saara Hellström, Miika Oinonen, Anna Salmela, Douglas Biber, Jesse Egbert, Sampo Pyysalo, Veronika Laippala
| Challenge: | Existing studies on register classification for web documents have limited results due to skewed datasets and low performance. |
| Approach: | They propose two new register-annotated corpora for French and Swedish . they show that deep pre-trained language models perform strongly in these languages . |
| Outcome: | The proposed models outperform existing models in English and Finnish and can match or surpass existing models. |
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT) (2025.acl-long)
Copied to clipboard
Laurie Burchell, Ona De Gibert Bonet, Nikolay Arefyev, Mikko Aulamo, Marta Bañón, Pinzhen Chen, Mariia Fedorova, Liane Guillou, Barry Haddow, Jan Hajič, Jindřich Helcl, Erik Henriksson, Mateusz Klimaszewski, Ville Komulainen, Andrey Kutuzov, Joona Kytöniemi, Veronika Laippala, Petter Mæhlum, Bhavitvya Malik, Farrokh Mehryary, Vladislav Mikhailov, Nikita Moghe, Amanda Myntti, Dayyán O’Brien, Stephan Oepen, Proyag Pal, Jousia Piha, Sampo Pyysalo, Gema Ramírez-Sánchez, David Samuel, Pavel Stepachev, Jörg Tiedemann, Dušan Variš, Tereza Vojtěchová, Jaume Zaragoza-Bernabeu
| Challenge: | a large number of textual data is needed to train state-of-the-art large language models. |
| Approach: | They propose a collection of monolingual and parallel corpora from the Internet Archive . they document the entire data pipeline and release the code to reproduce it . |
| Outcome: | The proposed collection of monolingual and parallel corpora is based on the HPLT v2 dataset . it includes 8T tokens covering 193 languages and 380M sentence pairs covering 51 languages . |
Explaining Classes through Stable Word Attributions (2022.findings-acl)
Copied to clipboard
| Challenge: | Input saliency methods have become popular for explaining predictions of deep learning models, but there has been little work investigating methods for aggregating prediction-level explanations to the class level. |
| Approach: | They propose a method to aggregate prediction-level explanations to the class level using XLM-R and Integrated Gradients input attribution methods. |
| Outcome: | The proposed method extracts keyword lists of classes from text classification tasks and evaluates them on web register data. |
Building Question-Answer Data Using Web Register Identification (2024.lrec-main)
Copied to clipboard
| Challenge: | Recent advances in web register (genre) identification have created a shortage of QA datasets for English and Finnish. |
| Approach: | They propose a machine learning-based method for extracting QA pairs from web-scale data using XLM-R and a multilingual CORE web register corpus . they then develop a NER-style token classifier to identify the QA text spans within these documents. |
| Outcome: | The proposed method is adaptable to any language given the availability of language models and extensive web data, but it is limited to English and Finnish. |
A Broad-coverage Corpus for Finnish Named Entity Recognition (2020.lrec-1)
Copied to clipboard
| Challenge: | Named entity recognition (NER) is a fundamental task in natural language processing (NLP). |
| Approach: | They propose to annotate Finnish named entity names using a new corpus built on the Universal Dependencies corpus. |
| Outcome: | The new annotation identifies over 10,000 mentions and maintains compatibility with a previously released single-domain corpus for Finnish NER. |
FinGPT: Large Generative Models for a Small Language (2023.emnlp-main)
Copied to clipboard
Risto Luukkonen, Ville Komulainen, Jouni Luoma, Anni Eskelinen, Jenna Kanerva, Hanna-Mari Kupari, Filip Ginter, Veronika Laippala, Niklas Muennighoff, Aleksandra Piktus, Thomas Wang, Nouamane Tazi, Teven Scao, Thomas Wolf, Osma Suominen, Samuli Sairanen, Mikko Merioksa, Jyrki Heinonen, Aija Vahtola, Samuel Antao, Sampo Pyysalo
| Challenge: | Neural language models excel in many tasks in NLP but are limited to smaller languages. |
| Approach: | They propose two approaches to pretrain large language models for Finnish . they train seven monolingual models from scratch and use Finnish as pretraining data . |
| Outcome: | The proposed model is based on a dataset of Finnish web crawls, news, social media and eBooks. |