Papers by Jörg Frohberg
CRASS: A Novel Data Set and Benchmark to Test Counterfactual Reasoning of Large Language Models (2022.lrec-1)
Copied to clipboard
| Challenge: | CRASS data set and benchmark provide novel test scheme to evaluate large language models . authors present and explain the CRAS data set, a novel basis to test reasoning and natural language understanding of LLMs . |
| Approach: | They introduce a new test scheme utilizing questionized counterfactual conditionals to evaluate large language models. |
| Outcome: | The proposed model sets out to be the most powerful and valid tool to evaluate large language models. |