Papers by Kawshik Manikantan
IdentifyMe: A Challenging Long-Context Mention Resolution Benchmark for LLMs (2025.naacl-short)
Copied to clipboard
| Challenge: | Recent evaluations of LLMs on coreference resolution have revealed that traditional output formats and evaluation metrics do not fully capture the models’ referential understanding. |
| Approach: | They propose a benchmark for mention resolution presented in a multiple-choice question format and a curated mixture of different mention types and corresponding entities. |
| Outcome: | The proposed model achieves 81.9% accuracy while the open model achieve 80%. |