Papers by Guillermo Marco
A Web Portal about the State of the Art of NLP Tasks in Spanish (2024.lrec-main)
Copied to clipboard
Enrique Amigó, Jorge Carrillo-de-Albornoz, Andrés Fernández, Julio Gonzalo, Guillermo Marco, Roser Morante, Laura Plaza, Jacobo Pedrosa
| Challenge: | a web portal has been created with information about the state of the art of natural language processing tasks in Spanish. |
| Approach: | They propose a web portal that provides information about the state of the art of natural language processing tasks in Spanish. |
| Outcome: | The portal provides information about forums, competitions, tasks and datasets in Spanish that would otherwise be spread in multiple articles and web sites. |
Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs (2025.coling-main)
Copied to clipboard
| Challenge: | a fine-tuned small language model (SLM) can generate human-like text, but it requires immense computational resources and large datasets. |
| Approach: | They evaluate the creative writing abilities of a fine-tuned small language model, BART-large . they compare it to human writers and two large language models: GPT-3.5 and GPT-4o . |
| Outcome: | The proposed model outperforms human writers and two large language models in two experiments . the results highlight how model size and fine-tuning influence creativity, fluency, and coherence . |
Pron vs Prompt: Can Large Language Models already Challenge a World-Class Fiction Author at Creative Text Writing? (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have recently shown strong competences generating human-like text, and in particular in short creative writing tasks. |
| Approach: | They conducted a contest between a novelist and a top performing LLM to determine whether they are ready to compete in creative writing skills with a human creative writer. |
| Outcome: | The results show that LLMs are far from challenging a top human creative writer. |
The Reader is the Metric: How Textual Features and Reader Profiles Explain Conflicting Evaluations of AI Creative Writing (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent studies comparing AI-generated and human-authored literary texts have produced conflicting results. |
| Approach: | They hypothesize that differences in reading quality can be explained by genuine differences in how readers interpret and value literature . |
| Outcome: | The authors show that the differences in reading quality are largely explained by differences in how readers interpret and value literature, rather than by an intrinsic quality of the texts evaluated. |
Bilingual Evaluation of Language Models on General Knowledge in University Entrance Exams with Minimal Contamination (2025.coling-main)
Copied to clipboard
Eva Sánchez Salido, Roser Morante, Julio Gonzalo, Guillermo Marco, Jorge Carrillo-de-Albornoz, Laura Plaza, Enrique Amigo, Andrés Fernandez García, Alejandro Benito-Santos, Adrián Ghajari Espinosa, Victor Fresno
| Challenge: | Existing benchmarks for Large Language Models have been proposed as single-task evaluations, but they are not fully comprehensive. |
| Approach: | They present a bilingual dataset that contains 1003 multiple-choice questions in Spanish and English. |
| Outcome: | The proposed model ranking is almost identical to the one obtained with MMLU . |