Papers by Dan Moreno
On the Challenges of Evaluating Compositional Explanations in Multi-Hop Inference: Relevance, Completeness, and Expert Ratings (2021.emnlp-main)
Copied to clipboard
| Challenge: | a large corpus of domain-expert relevance ratings augments a corpus for compositional explanations . a writer's study shows that the evaluations of compositional inference models underestimate performance . |
| Approach: | They construct a corpus of 126k domain-expert relevance ratings that augment explanations to standardized science exam questions. |
| Outcome: | The results show that evaluations underestimate performance of compositional explanations . they show that models regularly discover and produce valid explanations that are different than gold explanations. |