Papers by Ally Zhang
Evaluating Models’ Local Decision Boundaries via Contrast Sets (2020.findings-emnlp)
Copied to clipboard
Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
| Challenge: | Standard test sets for supervised learning evaluate in-distribution generalization but are misleading when a dataset has systematic gaps. |
| Approach: | They propose a more rigorous annotation paradigm for NLP that helps to close systematic gaps in the test data. |
| Outcome: | The proposed model performs significantly lower on contrast sets than on the original test sets—up to 25% in some cases. |
A Corpus for Reasoning about Natural Language Grounded in Photographs (P19-1)
Copied to clipboard
| Challenge: | a dataset for visual reasoning with natural language and images is available. |
| Approach: | They propose a dataset for joint reasoning about natural language and images . they crowdsource 107,292 examples of English sentences paired with web photographs . |
| Outcome: | The proposed dataset combines 107,292 examples of English sentences with web photographs . Qualitative analysis shows the data requires compositional joint reasoning . |