Papers by Xabier Saralegi
DIPLomA: Efficient Adaptation of Instructed LLMs to Low-Resource Languages via Post-Training Delta Merging (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Using delta-based weight merging, open-weight instruction-tuned large language models can be efficiently adapted to low-resource languages without costly large-scale post-training. |
| Approach: | They propose a delta-based instruction-preserving language adaptation strategy that decouples adaptation from post-training alignment by pretraining a foundational LLM on monolingual target-language data and then injecting instruction-following capabilities via delta-derived weight merging. |
| Outcome: | The proposed method achieves significant gains in instruction-following, linguistic proficiency, and safety on Basque, Welsh, and Swahili while preserving the original model's multilingual performance. |
TANDO: A Corpus for Document-level Machine Translation (2022.lrec-1)
Copied to clipboard
Harritxu Gete, Thierry Etchegoyhen, David Ponce, Gorka Labaka, Nora Aranberri, Ander Corral, Xabier Saralegi, Igor Ellakuria, Maite Martin
| Challenge: | Document-level Neural Machine Translation aims to increase the quality of neural translation models by taking into account contextual information. |
| Approach: | They propose to use document-level corpus for Basque-Spanish language pairs to take into account contextual information and perform fine-grained evaluations of gender and gender. |
| Outcome: | The proposed corpus is suitable for fine-grained evaluation of document-level machine translation systems. |
How Well Can BERT Learn the Grammar of an Agglutinative and Flexible-Order Language? The Case of Basque. (2024.lrec-main)
Copied to clipboard
| Challenge: | Neural Language Models (NLMs) have demonstrated effectiveness in acquiring skills related to human language use. |
| Approach: | They hypothesize that languages with complex grammar present substantial challenges during the pre-training phase . they constructed a test set that measures grammatical knowledge of BERT models trained under various pre-training configurations using corpus size, model size, number of epochs, and lemmatization. |
| Outcome: | The proposed model is based on a student-based minimal pairs test set with a grammatically correct and an incorrect sentence. |
Morphology Aware Source Term Masking for Terminology-Constrained NMT (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent research in terminology-constrained NMT systems focuses on data-driven approaches to generating translations. |
| Approach: | They propose a method that appends target term lemmas to their corresponding source terms in the input sentence while retaining essential grammatical information. |
| Outcome: | The proposed method improves on the “copy-and-inflect” method in two translation directions with different levels of source morphological complexity. |
BasqBBQ: A QA Benchmark for Assessing Social Biases in LLMs for Basque, a Low-Resource Language (2025.coling-main)
Copied to clipboard
| Challenge: | Existing pre-trained language models can propagate social biases in under-resourced languages like Basque. |
| Approach: | They propose a benchmark to assess biases in Basque using a multiple-choice question-answering task. |
| Outcome: | The proposed dataset is the first to assess biases in Basque across eight domains . larger models achieve better accuracy, but ambiguous cases remain challenging . |
Pipeline Analysis for Developing Instruct LLMs in Low-Resource Languages: A Case Study on Basque (2025.naacl-long)
Copied to clipboard
| Challenge: | Large language models are typically optimized for resource-rich languages like English . however, the proprietary nature of these models makes them impractical for many researchers and developers. |
| Approach: | They propose to develop large language models that can follow instructions in Basque . they focus on three key stages: pre-training, instruction tuning, and alignment with human preferences . |
| Outcome: | The proposed models improve natural language understanding (NLU) of the foundational model by 12 points . the results show that the models can follow instructions in Basque with human preferences . |
Scaling Laws for BERT in Low-Resource Settings (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models require huge training corpora, which is unobtainable for most NLP practitioners. |
| Approach: | They propose power-law formulas that relate model size, corpora size and computation power to find the optimal settings in advance given a fixed budget. |
| Outcome: | The proposed models perform better on MLM and NLU tasks on four languages of different linguistic characteristics. |
BasqueGLUE: A Natural Language Understanding Benchmark for Basque (2022.lrec-1)
Copied to clipboard
| Challenge: | Natural Language Understanding (NLU) benchmarks are costly to develop and language-dependent . basqueGLUE is the first benchmark for Basque, a less-resourced language . |
| Approach: | They propose a benchmark for Basque, a less-resourced language, using existing datasets. |
| Outcome: | The proposed benchmarks take into account a wide and diverse set of NLU tasks that require some form of language understanding beyond the detection of superficial clues. |
Not Enough Data to Pre-train Your Language Model? MT to the Rescue! (2023.findings-acl)
Copied to clipboard
| Challenge: | In recent years, transformer-based language models (LMs) have become the default approach for many NLP tasks. |
| Approach: | They compare the performance of transformer-based language models with machine-translated corpora. |
| Outcome: | The proposed model can be improved with real data, but further research is needed. |
Give your Text Representation Models some Love: the Case for Basque (2020.lrec-1)
Copied to clipboard
Rodrigo Agerri, Iñaki San Vicente, Jon Ander Campos, Ander Barrena, Xabier Saralegi, Aitor Soroa, Eneko Agirre
| Challenge: | Word embeddings and pre-trained language models are expensive to train and are often used by small companies and research groups to build their own. |
| Approach: | They propose to use word embeddings and pre-trained language models to build rich representations of text and improve NLP tasks. |
| Outcome: | The proposed models perform better than publicly available versions in downstream NLP tasks for Basque. |
XNLIeu: a dataset for cross-lingual NLI in Basque (2024.naacl-long)
Copied to clipboard
| Challenge: | XNLI is a popular benchmark used to evaluate cross-lingual Natural Language Understanding (NLU) in languages such as English, Basque and other low-resource languages. |
| Approach: | They expand XNLI to include Basque, a low-resource language that can benefit from transfer-learning approaches. |
| Outcome: | The proposed dataset includes Basque, a low-resource language that can benefit from transfer-learning approaches. |
Building a Task-oriented Dialog System for Languages with no Training Data: the Case for Basque (2020.lrec-1)
Copied to clipboard
| Challenge: | a new approach to task-oriented dialog systems is proposed for less-resourced languages . the authors compare training data projection with direct model transfer strategies . |
| Approach: | They propose to project existing annotations in rich-resource languages and use NMT to project them. |
| Outcome: | The proposed system outperforms models trained on Basque projected data in intent classification and slot filling tasks. |