Papers by Michael Kirchhof
Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Results (2025.acl-short)
Copied to clipboard
Andrea Santilli, Adam Golinski, Michael Kirchhof, Federico Danieli, Arno Blaas, Miao Xiong, Luca Zappella, Sinead Williamson
| Challenge: | Language Models (LMs) produce factually incorrect outputs, or "hallucinations" Xiao and Wang et al., 2023) rely on AUROC to assess how well UQ methods distinguish correct from incorrect output. |
| Approach: | They propose to use length biases in correctness functions to skew UQ evaluations . they propose to employ LM-as-a-judge methods as the least length-biased . |
| Outcome: | The proposed method is least length-biased, offering a promising path for a fairer evaluation. |