Papers by Claire Barale
LexTime: A Benchmark for Temporal Ordering of Legal Events (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks lack specialized language evaluation for LLMs on event ordering in legal contexts. |
| Approach: | They propose to use a dataset to evaluate LLMs' event ordering capabilities in legal language to evaluate their temporal relations with legal events. |
| Outcome: | The proposed model improves on a dataset of 512 instances from U.S. Federal Complaints with annotated event pairs and their temporal relations. |
Are We Done with MMLU? (2025.naacl-long)
Copied to clipboard
Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino, Rohit Saxena, Xuanli He, Yu Zhao, Xiaotang Du, Mohammad Reza Ghasemi Madani, Claire Barale, Robert McHardy, Joshua Harris, Jean Kaddour, Emile Van Krieken, Pasquale Minervini
| Challenge: | MMLU is widely adopted but its ground truth errors obscure the true capabilities of LLMs. |
| Approach: | They propose a framework for identifying dataset errors using a novel error annotation protocol and a subset of 5,700 manually re-annotated questions. |
| Outcome: | The proposed framework is based on 5,700 re-annotated questions from the MMLU benchmark. |
Automated Refugee Case Analysis: A NLP Pipeline for Supporting Legal Practitioners (2023.findings-acl)
Copied to clipboard
| Challenge: | In Canada, retrieving similar cases and their analysis is a key part of legal work . long processing times are due to a significant backlog and to the amount of work required from counsels . |
| Approach: | They propose to extend existing neural named-entity recognition models to retrieve 19 categories of items from refugee cases. |
| Outcome: | The proposed pipeline achieves a superior F1- score on five of the targeted categories and superior to 80% on an additional 4 categories. |