Papers by Martin Preiß
ReFACT: A Benchmark for Scientific Confabulation Detection with Positional Error Annotations (2026.eacl-long)
Copied to clipboard
Yindong Wang, Martin Preiß, Margarita Bugueño, Jan Vincent Hoffbauer, Abdullatif Ghajar, Tolga Buz, Gerard de Melo
| Challenge: | Evaluating 9 state-of-the-art LLMs reveals two critical limitations: 61% of incorrect span predictions are semantically unrelated to actual errors. |
| Approach: | They propose a benchmark of 1,001 expert-annotated question-answer pairs with span-level error annotations derived from Reddit's r/AskScience. |
| Outcome: | Evaluating 9 state-of-the-art LLMs, we find that comparative judgment is paradoxically harder than independent detection when comparing answers side-by-side. |