Papers by Athiya Deviyani
Contextual Metric Meta-Evaluation by Measuring Local Metric Accuracy (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to metric meta-evaluation focus on general statements about absolute and relative quality of metrics across arbitrary system outputs, but in practice, metrics are applied in highly contextual settings. |
| Approach: | They propose a method for contextual metric meta-evaluation by comparing local metric accuracy. |
| Outcome: | The proposed method compares the local metric accuracy of evaluation metrics across translation, speech recognition, and ranking tasks. |