Papers by Irune Zubiaga
La Leaderboard: A Large Language Model Leaderboard for Spanish Varieties and Languages of Spain and Latin America (2025.acl-long)
Copied to clipboard
María Grandury, Javier Aula-Blasco, Júlia Falcão, Clémentine Fourrier, Miguel González Saiz, Gonzalo Martínez, Gonzalo Santamaria Gomez, Rodrigo Agerri, Nuria Aldama García, Luis Chiruzzo, Javier Conde, Helena Gomez Adorno, Marta Guerrero Nieto, Guido Ivetta, Natàlia López Fuertes, Flor Miriam Plaza-del-Arco, María-Teresa Martín-Valdivia, Helena Montoro Zamorano, Carmen Muñoz Sanz, Pedro Reviriego, Leire Rosado Plaza, Alejandro Vaca Serrano, Estrella Vallecillo-Rodríguez, Jorge Vallego, Irune Zubiaga
| Challenge: | La Leaderboard is the first open-source leaderboard to evaluate generative Large Language Models (LLMs) in languages and language varieties of Spain and Latin America. |
| Approach: | They propose to use La Leaderboard to evaluate generative Large Language Models in Spanish and Latin America. |
| Outcome: | La Leaderboard is the first open-source leaderboard to evaluate generative LLMs in languages and language varieties of Spain and Latin America. |
A LLM-based Ranking Method for the Evaluation of Automatic Counter-Narrative Generation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for evaluating CNs are expensive, time-consuming, and subjective, but lack a universal truth and the lack of a 'universal truth' . |
| Approach: | They propose a model ranking pipeline based on pairwise comparisons of generated CNs from different models organized in a tournament-style format to improve the evaluation process. |
| Outcome: | The proposed method achieves a high correlation with human preference, with a score of 0.88, and compares chat, instruct, and base models, exploring their strengths and limitations. |