Papers by Rodrigo Agerri
Building Named Entity Recognition Taggers via Parallel Corpora (L18-1)
Copied to clipboard
| Challenge: | Existing methods to generate semantic processors for languages lacking hand curated data are inefficiently slow and unaffordable in terms of human resources and economic costs. |
| Approach: | They propose to use statistical word alignments to project annotations from multiple sources to a target language. |
| Outcome: | The proposed method is effective to transport NER annotations across languages . it can generate a good statistical model for a new target language . |
Does Corpus Quality Really Matter for Low-Resource Languages? (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on multilingual pre-training has relied on automatically filtered versions of CommonCrawl. |
| Approach: | They propose to use tailored crawling to identify and scrape websites with high-quality content to improve representation learning in Basque. |
| Outcome: | The proposed corpus, called EusCrawl, has a much higher quality according to native annotators than the Basque portion of popular multilingual corpora like CC100 and mC4. |
MedMT5: An Open-Source Multilingual Text-to-Text LLM for the Medical Domain (2024.lrec-main)
Copied to clipboard
Iker García-Ferrero, Rodrigo Agerri, Aitziber Atutxa Salazar, Elena Cabrio, Iker de la Iglesia, Alberto Lavelli, Bernardo Magnini, Benjamin Molinet, Johana Ramirez-Romero, German Rigau, Jose Maria Villa-Gonzalez, Serena Villata, Andrea Zaninello
| Challenge: | Existing studies on large language models for medical applications have focused on a single language . medical mT5 outperforms both encoders and similar sized text-to-text models in English, French, and Italian benchmarks . |
| Approach: | They propose to train Medical mT5, the first open-source text-to-text multilingual model for the medical domain. |
| Outcome: | The proposed model outperforms encoders and similar sized models on the Spanish, French, and Italian benchmarks while being competitive with current state-of-the-art models in English. |
Argument Mining in Data Scarce Settings: Cross-lingual Transfer and Few-shot Techniques (2024.acl-long)
Copied to clipboard
| Challenge: | Recent work on sequence labelling has explored different strategies to mitigate the lack of manually annotated data for the large majority of the world languages. |
| Approach: | They propose to use the mask objective to exploit the few-shot capabilities of pre-trained language models to improve their performance. |
| Outcome: | The proposed model-transfer outperforms data-transference and fine-tuning outperformed few-shot methods for Argument Mining task. |
Annotating Abstract Meaning Representations for Spanish (L18-1)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) is a semantic representation language for natural language processing. |
| Approach: | They propose a method that would lay the groundwork for building a large semantic bank for Spanish . they propose to use a database to annotate AMRs for other languages . |
| Outcome: | The proposed method would lay the groundwork for building a large semantic bank for Spanish and guide those who would like to implement it for other languages. |
Developing New Linguistic Resources and Tools for the Galician Language (L18-1)
Copied to clipboard
| Challenge: | Existing resources and tools for the Galician language are lacking for other less-resourced languages, such as statistical tools for lemmatization and Named Entity Recognition. |
| Approach: | They propose to develop a manually revised corpus for POS tagging and lemmatization, and a new manually annotated corpus to train existing statistical tools for the Galician language. |
| Outcome: | The proposed resources include a new corpus for POS tagging and lemmatization, and a manually annotated corpus to handle Named Entity recognition. |
Metaphor and Large Language Models: When Surface Features Matter More than Deep Understanding (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on metaphor processing have focused on single datasets and specific task settings, often using artificially constructed data through lexical replacement. |
| Approach: | They propose to evaluate the capabilities of Large Language Models (LLMs) in metaphor interpretation across multiple datasets, tasks, and prompt configurations. |
| Outcome: | The proposed frameworks are more realistic and efficient than current models and are more efficient than existing models. |
Truth Knows No Language: Evaluating Truthfulness Beyond English (2025.acl-long)
Copied to clipboard
Blanca Calvo Figueras, Eneko Sagarzazu, Julen Etxaniz, Jeremy Barnes, Pablo Gamallo, Iria de-Dios-Flores, Rodrigo Agerri
| Challenge: | a new benchmark evaluates the truthfulness of large language models (LLMs) based on imitative falsehoods. |
| Approach: | They propose a professionally translated extension of the TruthfulQA benchmark . it evaluates truthfulness in Basque, Catalan, Galician, and Spanish . |
| Outcome: | The proposed extension of the TruthfulQA benchmark evaluates truthfulness in Basque, Catalan, Galician, and Spanish. |
Evaluating Shortest Edit Script Methods for Contextual Lemmatization (2024.lrec-main)
Copied to clipboard
| Challenge: | Modern contextual lemmatizers often rely on automatically induced Shortest Edit Scripts (SES) supervised contextual methods are used to perform lemma classification tasks. |
| Approach: | They propose to use masked language encoders to compute shortest edit Scripts (SES) SES is the number of edit operations to transform a word form into its lemma . |
| Outcome: | The proposed model outperforms language-specific models in all evaluation settings with seven languages of different morphological complexity. |
Lemma Dilemma: On Lemma Generation Without Domain- or Language-Specific Training Data (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) can generate lemmas in context without prior fine-tuning. |
| Approach: | They compare in-context lemma generation with traditional fully supervised approaches . they use encoder-only supervised methods and cross-lingual methods . |
| Outcome: | The proposed model outperforms the traditional fully supervised approach in the context of lemmatization tasks. |
Dynamic Knowledge Integration for Evidence-Driven Counter-Argument Generation with Large Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Argumentation in natural language processing (NLP) is becoming an indispensable tool in many application domains such as public policy, law, medicine, and education. |
| Approach: | They propose a reconstructed dataset of argument and counter-argument pairs . they propose integrating dynamic external knowledge from the web to improve counter-arguments . |
| Outcome: | The proposed method shows stronger correlation with human judgments compared to reference-based metrics. |
La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America (2025.acl-long)
Copied to clipboard
María Grandury, Javier Aula-Blasco, Júlia Falcão, Clémentine Fourrier, Miguel González Saiz, Gonzalo Martínez, Gonzalo Santamaria Gomez, Rodrigo Agerri, Nuria Aldama García, Luis Chiruzzo, Javier Conde, Helena Gomez Adorno, Marta Guerrero Nieto, Guido Ivetta, Natàlia López Fuertes, Flor Miriam Plaza-del-Arco, María-Teresa Martín-Valdivia, Helena Montoro Zamorano, Carmen Muñoz Sanz, Pedro Reviriego, Leire Rosado Plaza, Alejandro Vaca Serrano, Estrella Vallecillo-Rodríguez, Jorge Vallego, Irune Zubiaga
| Challenge: | La Leaderboard is the first open-source leaderboard to evaluate generative Large Language Models (LLMs) in languages and language varieties of Spain and Latin America. |
| Approach: | They propose to use La Leaderboard to evaluate generative Large Language Models in Spanish and Latin America. |
| Outcome: | La Leaderboard is the first open-source leaderboard to evaluate generative LLMs in languages and language varieties of Spain and Latin America. |
Multilingual Stance Detection in Tweets: The Catalonia Independence Corpus (2020.lrec-1)
Copied to clipboard
| Challenge: | stance detection is a method to determine the attitude of a text with respect to a specific topic or claim. |
| Approach: | They propose a multilingual dataset for stance detection in Twitter for the Catalan and Spanish languages. |
| Outcome: | The proposed dataset shows that it is well balanced for multilingual and cross-lingual research. |
Scaling Laws for BERT in Low-Resource Settings (2023.findings-acl)
Copied to clipboard
| Challenge: | Large language models require huge training corpora, which is unobtainable for most NLP practitioners. |
| Approach: | They propose power-law formulas that relate model size, corpora size and computation power to find the optimal settings in advance given a fixed budget. |
| Outcome: | The proposed models perform better on MLM and NLU tasks on four languages of different linguistic characteristics. |
BasqueGLUE: A Natural Language Understanding Benchmark for Basque (2022.lrec-1)
Copied to clipboard
| Challenge: | Natural Language Understanding (NLU) benchmarks are costly to develop and language-dependent . basqueGLUE is the first benchmark for Basque, a less-resourced language . |
| Approach: | They propose a benchmark for Basque, a less-resourced language, using existing datasets. |
| Outcome: | The proposed benchmarks take into account a wide and diverse set of NLU tasks that require some form of language understanding beyond the detection of superficial clues. |
The Interplay between Metaphors and NLP (2026.acl-tutorials)
Copied to clipboard
| Challenge: | This tutorial will provide an overview of the metaphor processing field. |
| Approach: | This tutorial will provide an overview of the metaphor processing field . it will focus on recent directions opened by LLMs for metaphor interpretation . |
| Outcome: | The tutorial will discuss the influence of various metaphor theories on the creation of annotated resources and models. |
T-Projection: High Quality Annotation Projection for Sequence Labeling Tasks (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Annotation projection is a task of transporting labels from source to target language without high-quality training data. |
| Approach: | They propose an annotation projection approach that leverages pretrained text2text models and machine translation technology to generate annotated data. |
| Outcome: | The proposed approach outperforms existing methods on intrinsic and extrinsic tasks in 5 Indo-European and 8 low-resource African languages. |
Benchmarking Critical Questions Generation: A Challenging Reasoning Task for Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Progress in the task of Critical Questions Generation has been hindered by the lack of suitable datasets and automatic evaluation standards. |
| Approach: | They propose a comprehensive approach to support the development and benchmarking of systems for this task. |
| Outcome: | The proposed approach supports the development and benchmarking of systems for this task. |
Benchmarking Meta-embeddings: What Works and What Does Not (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to build meta-embeddings have been evaluated using a variety of methods and datasets, which makes it difficult to draw meaningful conclusions regarding the merits of each approach. |
| Approach: | They propose a unified framework for a fair and objective meta-embedding evaluation using intrinsic and extrinsic tasks. |
| Outcome: | The proposed framework outperforms existing methods on intrinsic and extrinsic evaluation benchmarks and outperformed existing methods. |
BasqueParl: A Bilingual Corpus of Basque Parliamentary Transcriptions (2022.lrec-1)
Copied to clipboard
Nayla Escribano, Jon Ander Gonzalez, Julen Orbegozo-Terradillos, Ainara Larrondo-Ureta, Simón Peña-Fernández, Olatz Perez-de-Viñaspre, Rodrigo Agerri
| Challenge: | a new corpus of Basque parliamentary transcripts is released to study political discourse in contrasting languages . a corpus containing political discourses from public institutions can be used for computational social science research . |
| Approach: | They present a corpus from Basque parliamentary transcripts and enrich it with metadata related to relevant attributes of speakers and speeches. |
| Outcome: | The proposed corpus is characterized by heavy Basque-Spanish code-switching . it provides interesting insights about language use of political representatives across time, parties and gender . |
Give your Text Representation Models some Love: the Case for Basque (2020.lrec-1)
Copied to clipboard
Rodrigo Agerri, Iñaki San Vicente, Jon Ander Campos, Ander Barrena, Xabier Saralegi, Aitor Soroa, Eneko Agirre
| Challenge: | Word embeddings and pre-trained language models are expensive to train and are often used by small companies and research groups to build their own. |
| Approach: | They propose to use word embeddings and pre-trained language models to build rich representations of text and improve NLP tasks. |
| Outcome: | The proposed models perform better than publicly available versions in downstream NLP tasks for Basque. |
Model and Data Transfer for Cross-Lingual Sequence Labelling in Zero-Resource Settings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have proposed data-based cross-lingual transfer as an effective technique for cross-linguistic sequence labelling, but they have failed to perform well. |
| Approach: | They propose to use data-based cross-lingual transfer to train supervised models from a source language to unlabelled target languages. |
| Outcome: | The proposed techniques outperform data-based cross-lingual transfer approaches in a zero-shot setting. |
Basque and Spanish Counter Narrative Generation: Data Creation and Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Davidson et al.: hate speech is a growing media presence, but research on generating CNs has been limited . he says a new dataset for CN generation is available for basque and spanish . this dataset is based on a multilingual encoder-decoder model . |
| Approach: | They propose a new Basque and Spanish dataset for automatic CN generation . they use machine translation and professional post-edition to generate CNs in both languages . |
| Outcome: | The proposed datasets show that training on post-edited data improves generation over monolingual settings . similar results in zero-shot crosslingual evaluations show multilingual data augmentation outperforms training in English and Spanish . |
CasiMedicos-Arg: A Medical Question Answering Dataset Annotated with Explanatory Argumentative Structures (2024.emnlp-main)
Copied to clipboard
Ekaterina Sviridova, Anar Yeginbergen, Ainara Estarrona, Elena Cabrio, Serena Villata, Rodrigo Agerri
| Challenge: | Existing tools to aid residents in teaching medical doctors to explain decisions are a key objective of AI in education. |
| Approach: | They present a multilingual dataset for Medical Question Answering where doctors can annotate correct and incorrect diagnoses with argument components and argument relations. |
| Outcome: | The proposed dataset consists of 558 clinical cases with explanations in English, Spanish, French, Italian and annotated with argument components and argument relations. |
TextBI: An Interactive Dashboard for Visualizing Multidimensional NLP Annotations in Social Media Data (2024.eacl-demo)
Copied to clipboard
Maxime Masson, Christian Sallaberry, Marie-Noelle Bessagnet, Annig Le Parc Lacayrelle, Philippe Roose, Rodrigo Agerri
| Challenge: | TextBI is a generic dashboard designed to present multidimensional text annotations on large volumes of multilingual social media data. |
| Approach: | They propose a generic dashboard that presents multidimensional text annotations on large volumes of multilingual social media data in a user-friendly, interactive interface. |
| Outcome: | The proposed dashboard focuses on four core dimensions: spatial, temporal, thematic, and personal, and supports additional enrichment data such as sentiment and engagement. |
A LLM-based Ranking Method for the Evaluation of Automatic Counter-Narrative Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for evaluating CNs are expensive, time-consuming, and subjective, but lack a universal truth and the lack of a 'universal truth' . |
| Approach: | They propose a model ranking pipeline based on pairwise comparisons of generated CNs from different models organized in a tournament-style format to improve the evaluation process. |
| Outcome: | The proposed method achieves a high correlation with human preference, with a score of 0.88, and compares chat, instruct, and base models, exploring their strengths and limitations. |