Papers by Luca Gioffré
Right Answer, Wrong Score: Uncovering the Inconsistencies of LLM Evaluation in Multiple-Choice Question Answering (2025.findings-acl)
Copied to clipboard
| Challenge: | Multiple-choice question answering tasks are one of the most commonly used tasks for evaluating Large Language Models (LLMs). |
| Approach: | They analyze whether existing answer extraction methods are aligned with human judgment and how they are influenced by answer constraints in the prompt across different domains. |
| Outcome: | The proposed evaluation strategies can be inconsistent with human judgment, and can lead to inaccurate and misleading comparisons. |
LiteraryQA: Towards Effective Evaluation of Long-document Narrative QA (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing Question Answering systems are limited by noisy documents and flawed QA pairs. |
| Approach: | They propose a high-quality subset of NarrativeQA focused on literary works . they identify and correct low-quality QA samples while removing extraneous text . |
| Outcome: | The proposed subset of NarrativeQA is based on literary works. |