Papers by Mateusz Klimaszewski
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement (2025.coling-main)
Copied to clipboard
| Challenge: | Modular deep learning is the most effective way to lift the curse of multilinguality. |
| Approach: | They propose a method which enables training-free post-processing to address this limitation by adding learning to the language adapters and transitioning the framework from a multi-task to a multiple language setup. |
| Outcome: | The proposed method consistently improves baselines with significant gains, especially in the most challenging case of zero-shot application. |
Is Modularity Transferable? A Case Study through the Lens of Knowledge Distillation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to modularity are limited to the case of pre-trained modules in a pre-training language model. |
| Approach: | They propose a method that allows the transfer of pre-trained PEFT modules between incompatible PLMs without any change in the inference complexity. |
| Outcome: | The proposed method allows the transfer of modules between incompatible PLMs without any change in the inference complexity. |
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT) (2025.acl-long)
Copied to clipboard
Laurie Burchell, Ona De Gibert Bonet, Nikolay Arefyev, Mikko Aulamo, Marta Bañón, Pinzhen Chen, Mariia Fedorova, Liane Guillou, Barry Haddow, Jan Hajič, Jindřich Helcl, Erik Henriksson, Mateusz Klimaszewski, Ville Komulainen, Andrey Kutuzov, Joona Kytöniemi, Veronika Laippala, Petter Mæhlum, Bhavitvya Malik, Farrokh Mehryary, Vladislav Mikhailov, Nikita Moghe, Amanda Myntti, Dayyán O’Brien, Stephan Oepen, Proyag Pal, Jousia Piha, Sampo Pyysalo, Gema Ramírez-Sánchez, David Samuel, Pavel Stepachev, Jörg Tiedemann, Dušan Variš, Tereza Vojtěchová, Jaume Zaragoza-Bernabeu
| Challenge: | a large number of textual data is needed to train state-of-the-art large language models. |
| Approach: | They propose a collection of monolingual and parallel corpora from the Internet Archive . they document the entire data pipeline and release the code to reproduce it . |
| Outcome: | The proposed collection of monolingual and parallel corpora is based on the HPLT v2 dataset . it includes 8T tokens covering 193 languages and 380M sentence pairs covering 51 languages . |
EuroGEST: Investigating gender stereotypes in multilingual language models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large language models encode social biases, but most benchmarks for gender bias remain English-centric. |
| Approach: | They propose a dataset to measure gender-stereotypical reasoning in large language models across English and 29 European languages. |
| Outcome: | The proposed method is highly accurate across languages and strong in translations and gender labels. |
Is a Document Educational or Just Wikipedia-Style? — Pitfalls of Classifier-Based Quality Filtering (2026.acl-short)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are pre-trained on massive data corpora, and the quality of these corporales is one of the main factors in achieving stateof-the-art performance. |
| Approach: | They propose to use Wikipedia-style reformatting to alter a model's quality assessment and enable low-quality content to surpass filtering thresholds. |
| Outcome: | The proposed model would reverse filtering decision for approximately 7% of evaluated documents, thereby admitting content into the pre-training corpus that would otherwise have been excluded. |
Multilingual Data Filtering using Synthetic Data from Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that effective filters can be created by utilising Large Language Models to synthetically label data, which is then used to train smaller neural models for filtering purposes. |
| Approach: | They extend this approach to languages beyond English to train neural models for filtering purposes. |
| Outcome: | The proposed approach is effective at filtering parallel text for translation quality and filtering for domain specificity. |
COMBO: State-of-the-Art Morphosyntactic Analysis (2021.emnlp-demo)
Copied to clipboard
| Challenge: | COMBO is an end-to-end NLP system for accurate part-of-speech tagging, morphological analysis, and (enhanced) dependency parsing. |
| Approach: | They propose a fully neural NLP system for accurate part-of-speech tagging, morphological analysis, lemmatisation, and (enhanced) dependency parsing. |
| Outcome: | The proposed system predicts categorical morphosyntactic features whilst also exposes their vector representations, extracted from hidden layers. |