Papers by Fredrik Carlsson
Lessons Learned from GPT-SW3: Building the First Large-Scale Generative Language Model for Swedish (2022.lrec-1)
Copied to clipboard
Ariel Ekgren, Amaru Cuba Gyllensten, Evangelia Gogoulou, Alice Heiman, Severine Verlinden, Joey Öhman, Fredrik Carlsson, Magnus Sahlgren
| Challenge: | a prerequisite for building large-scale generative models for other languages is access to large amounts of high-quality text data and powerful computational resources. |
| Approach: | They present a 3.5 billion parameter autoregressive language model, trained on a 100 GB Swedish corpus. |
| Outcome: | The proposed model performs well on a 100 GB Swedish corpus and is competent in comparison with existing models of similar size. |
Cross-lingual and Multilingual CLIP (2022.lrec-1)
Copied to clipboard
| Challenge: | OpenAI released CLIP, a model that relates the textual and visual domains with unprecedented accuracy. |
| Approach: | They propose to use cross-lingual teacher learning to re-train an English textual encoder using a large dataset of images and captions. |
| Outcome: | The proposed method outperforms baselines on multilingual image-text retrieval while retaining low cost. |
Fine-Grained Controllable Text Generation Using Non-Residual Prompting (2022.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to control the text generation process are not expressive enough. |
| Approach: | They propose an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps. |
| Outcome: | The proposed architecture is expressive and versatile on multiple experimental settings. |
GPT-SW3: An Autoregressive Language Model for the Scandinavian Languages (2024.lrec-main)
Copied to clipboard
Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk, Joey Öhman, Tim Isbister, Evangelia Gogoulou, Fredrik Carlsson, Judit Casademont, Magnus Sahlgren
| Challenge: | a growing interest in building and applying large language models for languages other than English is fueling interest in developing LLMs for smaller languages. |
| Approach: | They describe the development process for the first native large generative language model for the North Germanic languages, GPT-SW3. |
| Outcome: | The proposed model is based on the generative language model for the North Germanic languages . it is a first-generation model with a high-quality data set and a low cost of implementation . |