Papers by Robert McHardy
Are We Done with MMLU? (2025.naacl-long)
Copied to clipboard
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, Pasquale Minervini
| Challenge: | MMLU is widely adopted but its ground truth errors obscure the true capabilities of LLMs. |
| Approach: | They propose a framework for identifying dataset errors using a novel error annotation protocol and a subset of 5,700 manually re-annotated questions. |
| Outcome: | The proposed framework is based on 5,700 re-annotated questions from the MMLU benchmark. |
Adversarial Training for Satire Detection: Controlling for Confounding Variables (N19-1)
Copied to clipboard
| Challenge: | Existing methods for satire detection focus on satirical news based on article sources . satiric news are written with the aim of mimicking regular news in diction . |
| Approach: | They propose a model for satire detection with an adversarial component to control for the confounding variable of publication source. |
| Outcome: | The proposed model improves generalization performance to unseen publications with an adversarial component. |