Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)
Copied to clipboard
| Challenge: | Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations. |
| Approach: | They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention. |
| Outcome: | The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets. |
Similar Papers
IIT-KGP at COIN 2019: Using pre-trained Language Models for modeling Machine Comprehension (D19-60)
Copied to clipboard
| Challenge: | Using pre-trained language models, we can model machine comprehension using commonsense reasoning. |
| Approach: | They propose a machine comprehension model that leverages pre-trained language models over commonsense knowledge bases. |
| Outcome: | The proposed model improves on baseline models and other commonsense knowledge bases. |
Commonsense Inference in Natural Language Processing (COIN) - Shared Task Report (D19-60)
Copied to clipboard
| Challenge: | The workshop on Commonsense Inference in NLP (COIN) evaluated text understanding systems' ability to draw inferences about facts that are not mentioned in the text, but that are assumed to be common ground. |
| Approach: | They propose to use commonsense knowledge to evaluate systems' ability to answer questions/queries about a text. |
| Outcome: | The proposed tasks evaluated systems in two contexts: Commonsense Inference and Commonsensible Inference. |
BLCU-NLP at COIN-Shared Task1: Stagewise Fine-tuning BERT for Commonsense Inference in Everyday Narrations (D19-60)
Copied to clipboard
| Challenge: | Experimental results show that our system achieves significant improvements over the baseline systems with 84.2% accuracy on the official test dataset. |
| Approach: | They propose a system to inject more external knowledge into everyday narrations . they use a pre-trained BERT model to fine-tune on a machine reading comprehension dataset . |
| Outcome: | The proposed system achieves significant improvements over baseline systems with 84.2% accuracy on the official test dataset. |
KARNA at COIN Shared Task 1: Bidirectional Encoder Representations from Transformers with relational knowledge for machine comprehension with common sense (D19-60)
Copied to clipboard
| Challenge: | Using Bidirectional Encoder Representations from Transformers(BERT) and external relational knowledge from ConceptNet, we are able to achieve an accuracy of 73.3 % on the official test data. |
| Approach: | They propose a model that uses Bidirectional Encoder Representations from Transformers and ConceptNet to tackle the problem of commonsense inference in natural language processing. |
| Outcome: | The proposed model achieves 73.3 % accuracy on the official test data. |
Commonsense Knowledge Transfer for Pre-trained Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in pre-trained language models have transformed the landscape of natural language processing. |
| Approach: | They propose a framework to transfer commonsense knowledge stored in a neural commonsensing model to a general-purpose pre-trained language model. |
| Outcome: | Empirical results show that the proposed framework improves the model’s performance on downstream tasks that require commonsense reasoning. |
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
Pretrain-KGE: Learning Knowledge Representation from Pretrained Language Models (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding models suffer from limited knowledge representation due to sparse and noisy dataset annotations. |
| Approach: | They propose to use pretrained language models to enhance knowledge representation by leveraging world knowledge from pretrained models. |
| Outcome: | Extensive experiments show that the proposed framework can improve results over existing models. |
Few-shot Knowledge Graph-to-Text Generation with Pretrained Language Models (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing models for KG-to-text generation are based on pretrained language models. |
| Approach: | They propose to automatically generate a text that describes the facts in knowledge graph (KG) they leverage the excellent capacities of pretrained language models (PLMs) in language understanding and generation. |
| Outcome: | The proposed model outperforms all comparison methods on fully-supervised and fewshot settings. |
XLPT-AMR: Cross-Lingual Pre-Training via Multi-Task Learning for Zero-Shot AMR Parsing and Text Generation (2021.acl-long)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) research is limited and challenging for languages other than English. |
| Approach: | They propose a cross-lingual pre-training approach for AMR parsing and text generation . they use an English-to-English parallel dataset and a multi-task learning approach . |
| Outcome: | The proposed approach outperforms baseline pre-training methods on English parsing and text generation tasks. |
Advances in Pre-Training Distributed Word Representations (L18-1)
Copied to clipboard
| Challenge: | Pre-trained word representations are a building block of many Natural Language Processing and Machine Learning applications. |
| Approach: | They propose to combine known tricks and a set of publicly available pre-trained word vector representations to train high-quality representations. |
| Outcome: | The proposed models outperform the current state of the art on a number of tasks while maintaining a high training speed to scale to massive amount of data. |