Papers with mid-
What Context Features Can Transformer Language Models Use? (2021.acl-long)
Copied to clipboard
| Challenge: | Recent studies show that transformer-based language models benefit from conditioning on contexts of hundreds to thousands of previous tokens. |
| Approach: | They propose to use lexical and structural information to ablate usable information in transformer language models. |
| Outcome: | The proposed model improves when conditioning on contexts of thousands of previous tokens. |
Building a Data Infrastructure for a Mid-Resource Language: The Case of Catalan (2024.lrec-main)
Copied to clipboard
Aitor Gonzalez-Agirre, Montserrat Marimon, Carlos Rodriguez-Penagos, Javier Aula-Blasco, Irene Baucells, Carme Armentano-Oller, Jorge Palomar-Giner, Baybars Kulebi, Marta Villegas
| Challenge: | Aina Project aims to provide Catalan with the resources needed to keep its relevance in AI/NLP applications. |
| Approach: | They propose a set of strategies to consider when improving technology support for a mid- or low-resource language . they propose annotated datasets and a framework to make models ready to use . |
| Outcome: | The Aina Project aims to provide Catalan with the necessary resources to keep its relevance in AI/NLP-related industry and research. |