Papers by Pratik Patil
Precise Model Benchmarking with Only a Few Observations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Accurate evaluation of large language models is crucial for identifying their strengths and weaknesses. |
| Approach: | They propose an empirical Bayes estimator that balances direct and regression estimates for each subgroup separately, improving the precision of subgroup-level estimates of model performance. |
| Outcome: | The proposed model reduces the mean squared error by up to 50% on multiple datasets. |