Papers by Jorge Carrillo-de-Albornoz
A Web Portal about the State of the Art of NLP Tasks in Spanish (2024.lrec-main)
Copied to clipboard
Enrique Amigó, Jorge Carrillo-de-Albornoz, Andrés Fernández, Julio Gonzalo, Guillermo Marco, Roser Morante, Laura Plaza, Jacobo Pedrosa
| Challenge: | a web portal has been created with information about the state of the art of natural language processing tasks in Spanish. |
| Approach: | They propose a web portal that provides information about the state of the art of natural language processing tasks in Spanish. |
| Outcome: | The portal provides information about forums, competitions, tasks and datasets in Spanish that would otherwise be spread in multiple articles and web sites. |
An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimental Results (2020.acl-main)
Copied to clipboard
| Challenge: | Existing Ordinal Classification metrics ignore the ordering between items or assume additional information. |
| Approach: | They propose a Closeness Evaluation Measure for Ordinal Classification based on Measurement Theory and Information Theory. |
| Outcome: | The proposed metric captures quality aspects from different traditional tasks simultaneously. |
Evaluating Sequence Labeling on the basis of Information Theory (2025.acl-long)
Copied to clipboard
| Challenge: | Existing metric families focus on certain aspects of sequence labeling tasks. |
| Approach: | They propose a metric that measures how much information each token contributes depending on different aspects of the sequence. |
| Outcome: | The proposed metric can satisfy all properties simultaneously. |
Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination (2025.coling-main)
Copied to clipboard
Eva Sánchez Salido, Roser Morante, Julio Gonzalo, Guillermo Marco, Jorge Carrillo-de-Albornoz, Laura Plaza, Enrique Amigo, Andrés Fernandez García, Alejandro Benito-Santos, Adrián Ghajari Espinosa, Victor Fresno
| Challenge: | Existing benchmarks for Large Language Models have been proposed as single-task evaluations, but they are not fully comprehensive. |
| Approach: | They present a bilingual dataset that contains 1003 multiple-choice questions in Spanish and English. |
| Outcome: | The proposed model ranking is almost identical to the one obtained with MMLU . |