Do Nuclear Submarines Have Nuclear Captains? A Challenge Dataset for Commonsense Reasoning over Adjectives and Objects (D19-1)
Copied to clipboard
| Challenge: | a dataset of human judgments is used to test the ability to construct models with an understanding of commonsense knowledge. |
| Approach: | They crowdsource sentences that answer a question about adjectives and their transitivity . they build strong baselines for the task using a classification approach . |
| Outcome: | The proposed model outperforms word-level models on commonsense reasoning tasks. |
Similar Papers
Commonsense Reasoning for Natural Language Processing (2020.acl-tutorials)
Copied to clipboard
| Challenge: | In this tutorial, we will outline the various types of commonsense knowledge and discuss techniques to gather and represent commonsence knowledge. |
| Approach: | This tutorial will provide researchers with the critical foundations and recent advances in commonsense representation and reasoning. |
| Outcome: | This tutorial will outline the various types of commonsense and discuss techniques to gather and represent commonsence knowledge while highlighting the challenges specific to this type of knowledge (e.g., reporting bias). |
Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing (D19-60)
Copied to clipboard
| Challenge: | Workshop on Commonsense Inference in Natural Language Processing focuses on commonsense knowledge representation and application in NLP tasks. |
| Approach: | COIN is a workshop on commonsense inference in natural language processing . workshop included two shared tasks on reading comprehension using commonsensense knowledge . |
| Outcome: | the workshop focused on modeling commonsense knowledge and commonsensing in natural language processing tasks. |
Do language models have coherent mental models of everyday things? (2023.acl-long)
Copied to clipboard
| Challenge: | Psychologists and cognitive scientists hypothesize that humans develop mental models of the world, namely internal, conceptual representations of the environment which we base our decisions and actions on. |
| Approach: | They propose to add a constraint satisfaction layer to the LM's raw predictions to apply commonsense constraints to reduce incoherence. |
| Outcome: | The proposed extension removes inconsistencies and improves accuracy by 16-20%. |
Using Commonsense Knowledge to Answer Why-Questions (2022.emnlp-main)
Copied to clipboard
Yash Kumar Lal, Niket Tandon, Tanvi Aggarwal, Horace Liu, Nathanael Chambers, Raymond Mooney, Niranjan Balasubramanian
| Challenge: | Existing approaches to integrating commonsense knowledge into large language models are implicit and explicit. |
| Approach: | They analyze the effects of model size and methods of injecting knowledge into TellMeWhy datasets to determine what aspects of commonsense knowledge are available in large language models. |
| Outcome: | The largest models yield substantial improvements over base models, but the amount of improvement decreases with larger model size. |
Explain Yourself! Leveraging Language Models for Commonsense Reasoning (P19-1)
Copied to clipboard
| Challenge: | Empirical results indicate that we can effectively leverage language models for commonsense reasoning. |
| Approach: | They propose to use commonsense auto-generated explanations to train language models to generate explanations that can be used during training and inference in a commonsensense Auto-Generated Explanation framework. |
| Outcome: | Empirical results show that the proposed framework improves on the commonsenseQA task by 10%. |
Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (D18-2)
Copied to clipboard
| Challenge: | 77 submissions were received for the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) 4 of the 73 valid submissions received were either invalid or withdrawn by the authors. |
| Approach: | The volume contains papers from the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) 4 of the 77 submissions were either invalid or withdrawn by the authors. |
| Outcome: | The system demonstrations session included papers from the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) 4 of the 73 valid submissions were either invalid or withdrawn by the authors. |
Scalar Adjective Identification and Multilingual Ranking (2021.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on scalar adjective ranking have focused on English due to the availability of datasets for evaluation. |
| Approach: | They propose a binary classification task to examine the models’ ability to distinguish scalar from relational adjectives in English. |
| Outcome: | The proposed task compares the models' ability to distinguish scalar from relational adjectives in English using monolingual and multilingual models. |
Evaluating a Century of Progress on the Cognitive Science of Adjective Ordering (2023.tacl-1)
Copied to clipboard
| Challenge: | a new study examines the performance of cognitive hypotheses for adjective ordering in 32 languages . linguists and cognitive scientists have proposed an array of hypothese predicting adjective ordering . |
| Approach: | They compare the combined performance of existing adjective ordering proposals across 32 languages . they propose to use a baseline that reflects random chance accuracy and a higher baseline that measures idealized order . |
| Outcome: | The proposed hypotheses are compared with baselines in 32 languages and with random and idealized baselines. |
Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (2021.emnlp-demo)
Copied to clipboard
| Challenge: | . - (EN) |
| Approach: | . - (EN) |
| Outcome: | . - (EN) |
Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (2025.emnlp-demos)
Copied to clipboard
| Challenge: | Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing are now available online. |
| Approach: | EMNLP 2025 conference on empirical methods in natural language processing held in Suzhou, china, on November 4-9, 2025. 77 papers accepted for inclusion in proceedings, resulting in 38% acceptance rate. |
| Outcome: | Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing are published . the conference accepted 77 papers, with a 38% acceptance rate . |