Papers by Momoka Furuhashi
Are Checklists Really Useful for Automatic Evaluation of Generative Tasks? (2025.emnlp-main)
Copied to clipboard
| Challenge: | evaluators using large language models face ambiguous criteria and inconsistent evaluations. |
| Approach: | They investigate whether checklists should be used for all questions or selectively . they generate checklists using six methods and evaluate their effectiveness across eight models . |
| Outcome: | The proposed method improves evaluation performance in pairwise comparisons while ignoring human-written criteria. |