Papers by Jasivan Sivakumar
FERMAT: An Alternative to Accuracy for Numerical Reasoning (2023.acl-long)
Copied to clipboard
| Challenge: | Existing numerical reasoning models are too weak for downstream tasks like fact-checking . FERMAT evaluates models on number understanding, mathematical operations, and training dependency . |
| Approach: | They propose a multi-view evaluation set for numerical reasoning in English that evaluates models on key numerical reasoning aspects instead of reporting a single score on a whole dataset. |
| Outcome: | FERMAT evaluates models on number understanding, mathematical operations, and training dependency. |