Papers by Nikolay Arefyev
A New Massive Multilingual Dataset for High-Performance Language Technologies (2024.lrec-main)
Copied to clipboard
Ona de Gibert, Graeme Nail, Nikolay Arefyev, Marta Bañón, Jelmer van der Linde, Shaoxiong Ji, Jaume Zaragoza-Bernabeu, Mikko Aulamo, Gema Ramírez-Sánchez, Andrey Kutuzov, Sampo Pyysalo, Stephan Oepen, Jörg Tiedemann
| Challenge: | a new massive multilingual dataset is available for language modeling and machine translation training. |
| Approach: | They present a massive multilingual dataset using web crawls from the Internet Archive and CommonCrawl . they use open-source software tools and high-performance computing to acquire, manage and process large corpora . |
| Outcome: | The HPLT language resources is a massive multilingual dataset . it includes monolingual and bilingual corpora extracted from CommonCrawl and the Internet Archive . the results are published online at the journal journal cense4 . |
Enriching Word Usage Graphs with Cluster Definitions (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing word usage graphs lack human interpretability of senses. |
| Approach: | They propose to enrich existing word usage graphs with cluster labels functioning as sense definitions. |
| Outcome: | The proposed dataset matches the definitions chosen from WordNet by two baseline systems. |
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT) (2025.acl-long)
Copied to clipboard
Laurie Burchell, Ona De Gibert Bonet, Nikolay Arefyev, Mikko Aulamo, Marta Bañón, Pinzhen Chen, Mariia Fedorova, Liane Guillou, Barry Haddow, Jan Hajič, Jindřich Helcl, Erik Henriksson, Mateusz Klimaszewski, Ville Komulainen, Andrey Kutuzov, Joona Kytöniemi, Veronika Laippala, Petter Mæhlum, Bhavitvya Malik, Farrokh Mehryary, Vladislav Mikhailov, Nikita Moghe, Amanda Myntti, Dayyán O’Brien, Stephan Oepen, Proyag Pal, Jousia Piha, Sampo Pyysalo, Gema Ramírez-Sánchez, David Samuel, Pavel Stepachev, Jörg Tiedemann, Dušan Variš, Tereza Vojtěchová, Jaume Zaragoza-Bernabeu
| Challenge: | a large number of textual data is needed to train state-of-the-art large language models. |
| Approach: | They propose a collection of monolingual and parallel corpora from the Internet Archive . they document the entire data pipeline and release the code to reproduce it . |
| Outcome: | The proposed collection of monolingual and parallel corpora is based on the HPLT v2 dataset . it includes 8T tokens covering 193 languages and 380M sentence pairs covering 51 languages . |
Multilingual Substitution-based Word Sense Induction (2024.lrec-main)
Copied to clipboard
| Challenge: | Word Sense Induction is the task of finding senses of an ambiguous word . many approaches to WSI are language-specific and are not easily adaptable to new languages. |
| Approach: | They propose to use multilingual substitution-based WSI methods that generalize to any language supported by the underlying multilingual language model with minimal to no adaptation required. |
| Outcome: | The proposed methods perform on par with monolingual approaches on popular English datasets while being language-specific. |
Cross-lingual Named Entity List Search via Transliteration (2020.lrec-1)
Copied to clipboard
| Challenge: | a common way to adapt out-of-vocabulary words is a challenge in cross-lingual tasks . intrinsic evaluation, i.e comparison to a single gold standard, might not be appropriate in the task of transliteration due to its high variability. |
| Approach: | They propose to train Transformer-based multilingual transliteration models on 6 high- and 4 less-resourced languages and compare them with bilingual models. |
| Outcome: | The proposed model outperforms bilingual models on less-resourced languages. |
Always Keep your Target in Mind: Studying Semantics and Improving Performance of Neural Lexical Substitution (2020.coling-main)
Copied to clipboard
| Challenge: | Lexical substitution is a powerful technology used in various NLP applications . it generates plausible words that can replace a given word in a textual context . |
| Approach: | They propose to use a large-scale comparative study to compare lexical substitution methods . they compare existing and new methods using word sense induction datasets . |
| Outcome: | The proposed methods improve competitive results by incorporating information about the target word into the models. |
NB-MLM: Efficient Domain Adaptation of Masked Language Models for Sentiment Analysis (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-training Masked Language Models (MLMs) on massive datasets is expensive, but it is performed for each domain or task individually and is resource-demanding. |
| Approach: | They propose a method for more efficient adaptation that focuses on predicting words with large weights of the Naive Bayes classifier trained for the task at hand. |
| Outcome: | The proposed method improves sentiment analysis by focusing on predicting words with large weights of the Naive Bayes classifier trained for the task at hand. |