The Norwegian Colossal Corpus: A Text Corpus for Training Large Norwegian Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | Norwegian is one of many languages lacking sufficient textual data to train quality language models. |
| Approach: | They propose to release 49GB of clean Norwegian textual data containing over 7B words . they hope to foster the creation of better Norwegian language models and multilingual language models . |
| Outcome: | The Norwegian Colossal Corpus (NCC) contains 49GB of clean Norwegian textual data containing over 7B words. |
Similar Papers
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus (2021.emnlp-main)
Copied to clipboard
Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner
| Challenge: | Large text corpora are often introduced with minimal documentation . documenting collection process, composition, intended uses, and other are key for structured, task-specific datasets. |
| Approach: | They propose to document a dataset created by applying filters to a single snapshot of Common Crawl. |
| Outcome: | The proposed dataset shows that blocklist filtering removes text from minority individuals and patents. |
The Abkhaz National Corpus (L18-1)
Copied to clipboard
| Challenge: | Abkhaz National Corpus is a comprehensive and open, grammatically annotated text corpus . it is currently growing and is being extended to include all important texts written in the language . |
| Approach: | They propose to use the Abkhaz National Corpus to annotate Abkhhaz texts . the corpus is a comprehensive and open, grammatically annotated text corpus . |
| Outcome: | The proposed corpus is a grammatically annotated text corpus which makes the language accessible to scientific investigations from various perspectives. |
The DReaM Corpus: A Multilingual Annotated Corpus of Grammars for the World’s Languages (2020.lrec-1)
Copied to clipboard
| Challenge: | Until recently, language descriptions were available in paper form only, with indexes as the only search aid. |
| Approach: | They propose to digitize a multilingual corpus of language descriptions and annotate it with various meta, word, and text attributes to make searching and analysis easier and more useful. |
| Outcome: | The proposed corpus is searchable through a couple of well-established corpus infrastructures. |
NoReC: The Norwegian Review Corpus (L18-1)
Copied to clipboard
Erik Velldal, Lilja Øvrelid, Eivind Alexander Bergem, Cathrine Stadsnes, Samia Touileb, Fredrik Jørgensen
| Challenge: | The Norwegian Review Corpus is a dataset of full-text reviews from major news sources. |
| Approach: | This paper presents the Norwegian Review Corpus, created for document-level sentiment analysis. |
| Outcome: | The corpus comprises more than 35,000 full-text reviews from a range of different domains. |
NorNE: Annotating Named Entities for Norwegian (2020.lrec-1)
Copied to clipboard
| Challenge: | Using the annotations of the existing treebank, we have created a dataset for named entity recognition for Norwegian. |
| Approach: | They propose to create a manually annotated corpus of named entities for Norwegian . they propose to add named entity annotations to existing treebank . |
| Outcome: | The proposed dataset extends the annotation of the existing Norwegian Dependency Treebank. |
MAMMOTH: Massively Multilingual Modular Open Translation @ Helsinki (2024.eacl-demo)
Copied to clipboard
Timothee Mickus, Stig-Arne Grönroos, Joseph Attieh, Michele Boggia, Ona De Gibert, Shaoxiong Ji, Niki Andreas Loppi, Alessandro Raganato, Raúl Vázquez, Jörg Tiedemann
| Challenge: | a growing trend towards modularization is limiting the size and information that can be handled in large language models. |
| Approach: | They propose a framework for training massively multilingual modular machine translation systems at scale. |
| Outcome: | The proposed framework is adapted to train multilingual models at scale on NVIDIA GPUs. |
Facilitating Corpus Usage: Making Icelandic Corpora More Accessible for Researchers and Language Users (2020.lrec-1)
Copied to clipboard
| Challenge: | Gigaword corpus is a large text corpus used in natural language processing . large corpora are needed to achieve better performance in the field of NLP . |
| Approach: | They propose a set of tools to facilitate the use of the Icelandic Gigaword Corpus . they provide n-grams based on the corpus, and a variety of pre-trained word embeddings models . |
| Outcome: | The proposed tools facilitate the use of the Icelandic Gigaword corpus in the field of Natural Language Processing and other fields. |
GPT-SW3: An Autoregressive Language Model for the Scandinavian Languages (2024.lrec-main)
Copied to clipboard
Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk, Joey Öhman, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, Judit Casademont, Magnus Sahlgren
| Challenge: | a growing interest in building and applying large language models for languages other than English is fueling interest in developing LLMs for smaller languages. |
| Approach: | They describe the development process for the first native large generative language model for the North Germanic languages, GPT-SW3. |
| Outcome: | The proposed model is based on the generative language model for the North Germanic languages . it is a first-generation model with a high-quality data set and a low cost of implementation . |
The Norwegian Parliamentary Speech Corpus (2022.lrec-1)
Copied to clipboard
| Challenge: | the dataset contains recordings of meetings at the Norwegian parliament . it is the first publicly available dataset containing unscripted, Norwegian speech . |
| Approach: | the Norwegian Parliamentary Speech Corpus is a publicly available speech dataset . it contains recordings of meetings from the Norwegian parliament with orthographic transcriptions . the dataset is intended to fill a gap in the available unscripted speech data . |
| Outcome: | the dataset contains recordings of meetings at the Norwegian parliament with orthographic transcriptions in Norwegian Bokml and Norwegian Nynorsk. |
NLEBench+NorGLM: A Comprehensive Empirical Analysis and Benchmark Dataset for Generative Language Models in Norwegian (2024.emnlp-main)
Copied to clipboard
Peng Liu, Lemei Zhang, Terje Farup, Even Lauvrak, Jon Ingvaldsen, Simen Eide, Jon Atle Gulla, Zhirong Yang
| Challenge: | Norwegian is under-represented within the most impressive breakthroughs in NLP tasks. |
| Approach: | they investigate the impact of existing Norwegian language models on Norwegian generation tasks . they pre-trained 4 Norwegian Open Language Models from parameter scales and architectures . |
| Outcome: | The proposed benchmark evaluates the performance of language models on Norwegian generation tasks. |