Papers with Commonsense
Every Answer Matters: Evaluating Commonsense with Probabilistic Measures (2024.acl-long)
Copied to clipboard
Qi Cheng, Michael Boratko, Pranay Kumar Yelugam, Tim O’Gorman, Nalini Singh, Andrew McCallum, Xiang Li
| Challenge: | Existing commonsense evaluations are often posed as multiple-choice questions, allowing models to exploit systematic biases. |
| Approach: | They propose a generative task that evaluates common sense via multiple open-ended generations and a method that strongly correlates with human judgments. |
| Outcome: | The proposed method outperforms strong language model baselines on a dataset of human and machine common sense. |
Shortcutted Commonsense: Data Spuriousness in Deep Learning of Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | a recent study has found that commonsense reasoning models are learning transferable generalizations . commonsensibility is a human capacity that has been a core challenge to Artificial Intelligence since its inception. |
| Approach: | They conduct an analysis of benchmarks that involve commonsense reasoning . they find that most datasets experimented with are problematic . commonsensence is a quintessential human capacity . |
| Outcome: | The proposed model is able to perform well on commonsense reasoning tasks . the model is not learning transferable generalizations or taking advantage of shortcuts . |
Broaden the Vision: Geo-Diverse Visual Commonsense Reasoning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Generally, commonsense knowledge is correlated with culture and geographic locations and is only shared locally. |
| Approach: | They construct a Geo-Diverse Visual Commonsense Reasoning dataset to test vision-and-language models’ ability to understand cultural and geo-location-specific commonsense. |
| Outcome: | The proposed models perform better in non-Western regions including East Asia, South Asia, and Africa than in the Western regions. |