Proceedings of the First Workshop on Commonsense Inference in Natural Language Processing
Cracking the Contextual Commonsense Code: Understanding Commonsense Reasoning Aptitude of Deep Contextual Representations (D19-60)
Copied to clipboard
| Challenge: | Pretrained deep contextual representations have advanced the state-of-the-art on various commonsense NLP tasks, but we lack a concrete understanding of their capabilities. |
| Approach: | They investigate BERT's ability to encode various commonsense features in its embedding space, but are still deficient in many areas. |
| Outcome: | The proposed model improves performance on a downstream commonsense reasoning task while using minimal data. |
A Hybrid Neural Network Model for Commonsense Reasoning (D19-60)
Copied to clipboard
| Challenge: | a hybrid neural network (HNN) model for commonsense reasoning is proposed . it combines language models and semantic similarity models to achieve new state-of-the-art results . |
| Approach: | They propose a hybrid neural network model for commonsense reasoning . it combines a masked language model and a semantic similarity model . |
| Outcome: | The proposed model outperforms the WNLI, WSC and PDP60 benchmarks on three commonsense reasoning tasks. |
Towards Generalizable Neuro-Symbolic Systems for Commonsense Question Answering (D19-60)
Copied to clipboard
| Challenge: | Recent approaches on non-extractive commonsense QA show increased performance . attention-based injection seems to be preferable for knowledge integration . |
| Approach: | They propose to use attention-based injection to integrate knowledge into commonsense QA models. |
| Outcome: | The proposed methods show that attention-based injection is preferable for knowledge integration, and that the degree of domain overlap plays a crucial role in determining model success. |
When Choosing Plausible Alternatives, Clever Hans can be Clever (D19-60)
Copied to clipboard
| Challenge: | Pretrained language models have shown large improvements in the commonsense reasoning benchmark COPA, but recent work has identified superficial cues in benchmark datasets which are predictive of the correct answer. |
| Approach: | They propose an extension of COPA that does not suffer from easy-to-exploit single token cues and exploits them. |
| Outcome: | The proposed extension of COPA does not suffer from easy-to-exploit single token cues. |
Commonsense about Human Senses: Labeled Data Collection Processes (D19-60)
Copied to clipboard
| Challenge: | Existing methods for recognizing mentions of human senses in text are lacking in common sense knowledge acquisition. |
| Approach: | They propose to use machine learning to acquire labeled data to extract common sense relationships pertaining to sense perception concepts. |
| Outcome: | The proposed method is effective when used with standard machine learning models on the task of sense recognition in text. |
Extracting Common Inference Patterns from Semi-Structured Explanations (D19-60)
Copied to clipboard
| Challenge: | Multi-hop inference suffers from semantic drift, or the tendency for chains of reasoning to "drift"' to unrelated topics. |
| Approach: | They propose to extract large high-confidence multi-hop inference patterns from a corpus of explanations by abstracting large-scale structure from logical sentences. |
| Outcome: | The proposed method extracts large high-confidence multi-hop inference patterns from a “matter” subset of elementary science exam questions. |
Commonsense Inference in Natural Language Processing (COIN) - Shared Task Report (D19-60)
Copied to clipboard
| Challenge: | The workshop on Commonsense Inference in NLP (COIN) evaluated text understanding systems' ability to draw inferences about facts that are not mentioned in the text, but that are assumed to be common ground. |
| Approach: | They propose to use commonsense knowledge to evaluate systems' ability to answer questions/queries about a text. |
| Outcome: | The proposed tasks evaluated systems in two contexts: Commonsense Inference and Commonsensible Inference. |
KARNA at COIN Shared Task 1: Bidirectional Encoder Representations from Transformers with relational knowledge for machine comprehension with common sense (D19-60)
Copied to clipboard
| Challenge: | Using Bidirectional Encoder Representations from Transformers(BERT) and external relational knowledge from ConceptNet, we are able to achieve an accuracy of 73.3 % on the official test data. |
| Approach: | They propose a model that uses Bidirectional Encoder Representations from Transformers and ConceptNet to tackle the problem of commonsense inference in natural language processing. |
| Outcome: | The proposed model achieves 73.3 % accuracy on the official test data. |
IIT-KGP at COIN 2019: Using pre-trained Language Models for modeling Machine Comprehension (D19-60)
Copied to clipboard
| Challenge: | Using pre-trained language models, we can model machine comprehension using commonsense reasoning. |
| Approach: | They propose a machine comprehension model that leverages pre-trained language models over commonsense knowledge bases. |
| Outcome: | The proposed model improves on baseline models and other commonsense knowledge bases. |
Jeff Da at COIN - Shared Task: BIG MOOD: Relating Transformers to Explicit Commonsense Knowledge (D19-60)
Copied to clipboard
| Challenge: | Recent studies show that large-scale pre-training models can be effective for large datasets. |
| Approach: | They propose a method of integrating contextual embeddings with commonsense graph embeddINGs by preprocessing knowledge bases and aligning tokens between misaligned tokenization methods. |
| Outcome: | The proposed method achieves higher accuracy than BERT and scores highest without pretraining. |
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)
Copied to clipboard
| Challenge: | Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations. |
| Approach: | They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention. |
| Outcome: | The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets. |
BLCU-NLP at COIN-Shared Task1: Stagewise Fine-tuning BERT for Commonsense Inference in Everyday Narrations (D19-60)
Copied to clipboard
| Challenge: | Experimental results show that our system achieves significant improvements over the baseline systems with 84.2% accuracy on the official test dataset. |
| Approach: | They propose a system to inject more external knowledge into everyday narrations . they use a pre-trained BERT model to fine-tune on a machine reading comprehension dataset . |
| Outcome: | The proposed system achieves significant improvements over baseline systems with 84.2% accuracy on the official test dataset. |
Commonsense inference in human-robot communication (D19-60)
Copied to clipboard
| Challenge: | a gap exists in natural language understanding of commands between humans and machines. |
| Approach: | They propose a method for commonsense inference to transform high-level commands into action commands for robotic systems to execute. |
| Outcome: | The proposed method allows to build a knowledge base that consists of a large set of commonsense inferences. |
Diversity-aware Event Prediction based on a Conditional Variational Autoencoder with Reconstruction (D19-60)
Copied to clipboard
| Challenge: | Typical event sequences are important class of commonsense knowledge . previous work in event prediction uses sequence-to-sequence models . however, what can happen after a given event is usually diverse . |
| Approach: | They propose to incorporate a conditional variational autoencoder into seq2seq for its ability to represent diverse next events as a probabilistic distribution. |
| Outcome: | The proposed model outperforms deterministic models in terms of precision and recall . the proposed model is based on a conditional variational autoencoder . |
Can a Gorilla Ride a Camel? Learning Semantic Plausibility from Text (D19-60)
Copied to clipboard
| Challenge: | Existing work on modeling semantic plausibility has focused on physical plausability but distributional methods fail when tested in supervised settings. |
| Approach: | They propose to use large pretrained language models to model plausibility in supervised settings by extracting attested events from a large corpus and injecting explicit commonsense knowledge into a distributional model. |
| Outcome: | The proposed model is effective in modeling plausibility in a supervised setting. |
How Pre-trained Word Representations Capture Commonsense Physical Comparisons (D19-60)
Copied to clipboard
| Challenge: | Pre-trained word representations capture common sense on physical properties such as size and weight. |
| Approach: | They investigate whether pre-trained representations capture comparisons and find they have higher accuracy than previous approaches. |
| Outcome: | The proposed models learn a consistent ordering over all the objects in the comparisons. |