Papers by Andrey Kutuzov
NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark (2025.findings-acl)
Copied to clipboard
Vladislav Mikhailov, Tita Enstad, David Samuel, Hans Christian Farsethås, Andrey Kutuzov, Erik Velldal, Lilja Øvrelid
| Challenge: | NorEval is a new evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs). |
| Approach: | They propose a new evaluation suite for large-scale standardized benchmarking of Norwegian generative language models (LMs) NorEval consists of 24 high-quality human-created datasets, of which five are created from scratch. |
| Outcome: | The evaluation framework and materials are publicly available. |
Representing ELMo embeddings as two-dimensional text online (2021.eacl-demos)
Copied to clipboard
| Challenge: | ELMoViz module adds support for contextualized embedding architectures, in particular for token embeddable word models. |
| Approach: | They propose to add a module to the free and open-source WebVectors toolkit which provides lexical hyperlinks to word representations in static embedding models. |
| Outcome: | The ELMoViz module adds support for contextualized embedding architectures, in particular for ELMa models. |
A New Massive Multilingual Dataset for High-Performance Language Technologies (2024.lrec-main)
Copied to clipboard
Ona de Gibert, Graeme Nail, Nikolay Arefyev, Marta Bañón, Jelmer van der Linde, Shaoxiong Ji, Jaume Zaragoza-Bernabeu, Mikko Aulamo, Gema Ramírez-Sánchez, Andrey Kutuzov, Sampo Pyysalo, Stephan Oepen, Jörg Tiedemann
| Challenge: | a new massive multilingual dataset is available for language modeling and machine translation training. |
| Approach: | They present a massive multilingual dataset using web crawls from the Internet Archive and CommonCrawl . they use open-source software tools and high-performance computing to acquire, manage and process large corpora . |
| Outcome: | The HPLT language resources is a massive multilingual dataset . it includes monolingual and bilingual corpora extracted from CommonCrawl and the Internet Archive . the results are published online at the journal journal cense4 . |
Making Fast Graph-based Algorithms with Graph Metric Embeddings (P19-1)
Copied to clipboard
| Challenge: | Graph measures, such as node distances, are inefficient to compute. |
| Approach: | They propose a way to learn graph embeddings by using vector operations instead of a graph structure. |
| Outcome: | The proposed method outperforms other graph embeddings on word similarity and word sense disambiguation tasks. |
Enriching Word Usage Graphs with Cluster Definitions (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing word usage graphs lack human interpretability of senses. |
| Approach: | They propose to enrich existing word usage graphs with cluster labels functioning as sense definitions. |
| Outcome: | The proposed dataset matches the definitions chosen from WordNet by two baseline systems. |
Explaining novel senses using definition generation with open language models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | We apply definition generators based on open-weights large language models to create explanations of novel senses. |
| Approach: | They apply open-weights large language models to create explanations of novel senses using target word usages as input. |
| Outcome: | The proposed definition generators perform on par with decoder-only models. |
An Expanded Massive Multilingual Dataset for High-Performance Language Technologies (HPLT) (2025.acl-long)
Copied to clipboard
Laurie Burchell, Ona De Gibert Bonet, Nikolay Arefyev, Mikko Aulamo, Marta Bañón, Pinzhen Chen, Mariia Fedorova, Liane Guillou, Barry Haddow, Jan Hajič, Jindřich Helcl, Erik Henriksson, Mateusz Klimaszewski, Ville Komulainen, Andrey Kutuzov, Joona Kytöniemi, Veronika Laippala, Petter Mæhlum, Bhavitvya Malik, Farrokh Mehryary, Vladislav Mikhailov, Nikita Moghe, Amanda Myntti, Dayyán O’Brien, Stephan Oepen, Proyag Pal, Jousia Piha, Sampo Pyysalo, Gema Ramírez-Sánchez, David Samuel, Pavel Stepachev, Jörg Tiedemann, Dušan Variš, Tereza Vojtěchová, Jaume Zaragoza-Bernabeu
| Challenge: | a large number of textual data is needed to train state-of-the-art large language models. |
| Approach: | They propose a collection of monolingual and parallel corpora from the Internet Archive . they document the entire data pipeline and release the code to reproduce it . |
| Outcome: | The proposed collection of monolingual and parallel corpora is based on the HPLT v2 dataset . it includes 8T tokens covering 193 languages and 380M sentence pairs covering 51 languages . |
Interpretable Word Sense Representations via Definition Generation: The Case of Semantic Change Analysis (2023.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to semantic change analysis are limited in their interpretation power and lack of explanatory power. |
| Approach: | They propose to use specialised Flan-T5 language models to generate a definition for each usage and a specialised word sense model to generate the most prototypical definition. |
| Outcome: | The proposed representations outperform token or usage sentence embeddings in word-in-context semantic similarity judgements and are a promising type of lexical representation for NLP. |
Diachronic word embeddings and semantic shifts: a survey (C18-1)
Copied to clipboard
| Challenge: | Existing methods for tracing time-related semantic shifts with word embedding models lack the cohesion, common terminology and shared practices of more established areas of natural language processing. |
| Approach: | They propose several axes along which these methods can be compared and propose a framework for comparison. |
| Outcome: | The proposed methods are compared with existing methods and outline their main challenges and potential applications. |
Monolingual or Multilingual Instruction Tuning: Which Makes a Better Alpaca (2024.findings-eacl)
Copied to clipboard
| Challenge: | Foundational large language models (LLMs) can be instruction-tuned to perform open-domain question answering, facilitating applications like chat assistants. |
| Approach: | They employ a dataset and machine translations of it to form multilingual data and use it to tune LLMs. |
| Outcome: | The proposed model is on par or better than a model for each language, and multilingual tuning with downsampled data is as powerful and robust. |
Definition generation for lexical semantic change detection (2024.findings-acl)
Copied to clipboard
| Challenge: | a number of studies have attempted to bridge the gap between lexical semantic change detection and sense-based LSCD methods. |
| Approach: | They propose a sense distribution based LSCD method which uses contextualized word definitions as 'senses' they argue that the method preserves interpretability and allows to inspect the reasons behind a specific shift in terms of discrete definitions-as-sense. |
| Outcome: | The proposed method outperforms previous sense-based methods on five datasets and three languages and preserves interpretability and allows to inspect the reasons behind a specific shift in terms of discrete definitions-as-senses. |
Unsupervised Semantic Frame Induction using Triclustering (P18-2)
Copied to clipboard
| Challenge: | Recent work on frame-semantics has enabled the development of wide-coverage frame parsers using supervised learning. |
| Approach: | They propose to use dependency triples to perform unsupervised frame induction on a Web-scale corpus. |
| Outcome: | The proposed approach performs state-of-the-art on a FrameNet-derived dataset and performs on par with competitive methods on . verb class clustering task. |
RuSemShift: a dataset of historical lexical semantic change in Russian (2020.coling-main)
Copied to clipboard
| Challenge: | et al. (2017) lexical semantic change analysis is still mostly done for English because of limited resources. |
| Approach: | They present a large-scale manually annotated test set for semantic change modeling in Russian . they use DURel framework to annotate Russian for two long-term time periods . |
| Outcome: | The proposed model performs well, but there are still areas for improvement . the results are promising, but the authors say they need to improve . |
NorDiaChange: Diachronic Semantic Change Dataset for Norwegian (2022.lrec-1)
Copied to clipboard
| Challenge: | NorDiaChange is the first dataset of diachronic semantic change on the lexical level for Norwegian. |
| Approach: | They describe a manual annotation process for a new dataset of diachronic semantic change for Norwegian. |
| Outcome: | The proposed dataset covers the time periods related to pre- and post-war events, oil and gas discovery in Norway, and technological developments. |
Trained on 100 million words and still in shape: BERT meets British National Corpus (2023.findings-eacl)
Copied to clipboard
| Challenge: | masked language models are trained on ever larger corpora, but pre-training on a modestly-sized but representative, well-balanced, and publicly available corpus can reach better performance than the original BERT model. |
| Approach: | They propose an optimized LM architecture called LTG-BERT that can be used to train a competitive language model on a small and standardizable corpus. |
| Outcome: | The proposed architecture outperforms the original English BERT model on a representative, well-balanced and publicly available corpus. |
Word Sense Disambiguation for 158 Languages using Word Embeddings Only (2020.lrec-1)
Copied to clipboard
Varvara Logacheva, Denis Teslenko, Artem Shelmanov, Steffen Remus, Dmitry Ustalov, Andrey Kutuzov, Ekaterina Artemova, Chris Biemann, Simone Paolo Ponzetto, Alexander Panchenko
| Challenge: | Existing methods of disambiguation of word senses are based on knowledge bases, taxonomies, and other externally built resources. |
| Approach: | They propose a method that takes a pre-trained word embedding model and induces a fully-fledged word sense inventory for 158 languages. |
| Outcome: | The proposed model is based on a pre-trained word embedding model and induces a fully-fledged word sense inventory in 158 languages. |