Papers by Pride Kavumba
When Choosing Plausible Alternatives, Clever Hans can be Clever (D19-60)
Copied to clipboard
| Challenge: | Pretrained language models have shown large improvements in the commonsense reasoning benchmark COPA, but recent work has identified superficial cues in benchmark datasets which are predictive of the correct answer. |
| Approach: | They propose an extension of COPA that does not suffer from easy-to-exploit single token cues and exploits them. |
| Outcome: | The proposed extension of COPA does not suffer from easy-to-exploit single token cues. |
Are Prompt-based Models Clueless? (2022.acl-long)
Copied to clipboard
| Challenge: | Prompting has reduced the data requirement by reusing the language model head and formatting the task input to match the pre-training objective. |
| Approach: | They propose to examine whether few-shot prompt-based models exploit superficial cues by reusing the model head and formatting the input to match the pre-training objective. |
| Outcome: | The proposed models perform well on instances with superficial cues, but often outperform random accuracy on instances without superficial cuing. |
Prompting for explanations improves Adversarial NLI. Is this true? {Yes} it is {true} because {it weakens superficial cues} (2023.findings-eacl)
Copied to clipboard
| Challenge: | Explanation prompts are used to generate an explanation for a given input . they are also used to improve model performance on adversarial benchmarks . |
| Approach: | They propose to use explanation prompts to generate an explanation that supports a label . they argue that prompting for explanations weakens superficial cues . |
| Outcome: | The proposed explanation prompts outperform label-only prompts on adversarial benchmarks. |
COPA-SSE: Semi-structured Explanations for Commonsense Reasoning (2022.lrec-1)
Copied to clipboard
| Challenge: | Semi-structured explanations for Choice of Plausible Alternatives (COPA-SSE) are a crowdsourced dataset of 9,747 common sense explanations . |
| Approach: | They propose a semi-structured approach to explain Choice of Plausible Alternatives questions using a crowdsourced dataset of 9,747 common sense explanations with ConceptNet relations but freely written concepts. |
| Outcome: | The proposed explanations are geared towards commonsense reasoners operating on knowledge graphs and serve as a starting point for improving such systems. |
Improving Evidence Detection by Leveraging Warrants (D19-66)
Copied to clipboard
| Challenge: | Existing methods for extracting warrants from a corpus of arguments are lacking in argument detection. |
| Approach: | They propose to extract multiple warrants from an existing corpus of arguments and then aggregate them . they show that the method needs to be improved, but that it can still improve evidence detection. |
| Outcome: | The proposed method can improve the performance of evidence detection by analyzing arguments and aggregating them. |
Rubrik’s Cube: Testing a New Rubric for Evaluating Explanations on the CUBE dataset (2025.acl-long)
Copied to clipboard
Diana Galvan-Sosa, Gabrielle Gaudeau, Pride Kavumba, Yunmeng Li, Hongyi Gu, Zheng Yuan, Keisuke Sakaguchi, Paula Buttery
| Challenge: | Large-Language Models (LLMs) are increasingly being used in explanation generation tasks due to their unreliability. |
| Approach: | They propose a rubric and a dataset of 26k explanations written and quality-annotated using the rubric by humans and six open- and closed-source LLMs to test their proposed rubric. |
| Outcome: | The proposed rubric and CUBE dataset focuses on reasoning and language tasks and provides the necessary diversity to test it. |
Learning to Learn to be Right for the Right Reasons (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent work shows that models trained on held-out data perform poorly on hard instances . previous methods have resorted to manual methods of encouraging models not to overfit to superficial cues . |
| Approach: | They propose to explicitly learn a model that does well on both easy and hard tests . they use Choice of Plausible Alternatives and Commonsense Explanation to evaluate the model . |
| Outcome: | The proposed model performs well on easy and hard tests with superficial cues but performs poorly on hard ones without superficial cuings. |