Papers by William Berrios
LMUNIT: Fine-grained Evaluation with Natural Language Unit Tests (2025.findings-emnlp)
Copied to clipboard
Jon Saad-Falcon, Rajan Pathe Vivek, William Berrios, Nandita Shankar Naik, Matija Franklin, Bertie Vidgen, Amanpreet Singh, Douwe Kiela, Shikib Mehri
| Challenge: | Using natural language unit tests, language models are costly and noisy, and automated metrics provide only coarse, difficult-to-interpret signals. |
| Approach: | They propose a paradigm that decomposes response quality into explicit, testable criteria and a unified scoring model, LMUnit, which combines multi-objective training across preferences, direct ratings, and natural language rationales. |
| Outcome: | The proposed paradigm significantly improves inter-annotator agreement and enables more effective LLM development workflows. |