BLCU-NLP at COIN-Shared Task1: Stagewise Fine-tuning BERT for Commonsense Inference in Everyday Narrations (D19-60)
Copied to clipboard
| Challenge: | Experimental results show that our system achieves significant improvements over the baseline systems with 84.2% accuracy on the official test dataset. |
| Approach: | They propose a system to inject more external knowledge into everyday narrations . they use a pre-trained BERT model to fine-tune on a machine reading comprehension dataset . |
| Outcome: | The proposed system achieves significant improvements over baseline systems with 84.2% accuracy on the official test dataset. |
Similar Papers
IIT-KGP at COIN 2019: Using pre-trained Language Models for modeling Machine Comprehension (D19-60)
Copied to clipboard
| Challenge: | Using pre-trained language models, we can model machine comprehension using commonsense reasoning. |
| Approach: | They propose a machine comprehension model that leverages pre-trained language models over commonsense knowledge bases. |
| Outcome: | The proposed model improves on baseline models and other commonsense knowledge bases. |
KARNA at COIN Shared Task 1: Bidirectional Encoder Representations from Transformers with relational knowledge for machine comprehension with common sense (D19-60)
Copied to clipboard
| Challenge: | Using Bidirectional Encoder Representations from Transformers(BERT) and external relational knowledge from ConceptNet, we are able to achieve an accuracy of 73.3 % on the official test data. |
| Approach: | They propose a model that uses Bidirectional Encoder Representations from Transformers and ConceptNet to tackle the problem of commonsense inference in natural language processing. |
| Outcome: | The proposed model achieves 73.3 % accuracy on the official test data. |
Pingan Smart Health and SJTU at COIN - Shared Task: utilizing Pre-trained Language Models and Common-sense Knowledge in Machine Reading Tasks (D19-60)
Copied to clipboard
| Challenge: | Existing approaches to represent knowledge in the low-dimensional space are to leverage large-scale unsupervised text corpus to train fixed or contextual representations. |
| Approach: | They propose to leverage large-scale unsupervised text corpus to train fixed or contextual language representations and to express knowledge into a knowledge graph (KG) they incorporate distributional representations of a KG onto the representations from pre-trained language models, via simply concatenation or multi-head attention. |
| Outcome: | The proposed models outperform the other models on the COIN: COmmonsense INference in Natural Language Processing (COIN) Workshop datasets. |
On Commonsense Cues in BERT for Solving Commonsense Tasks (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models can capture syntactic features, semantic information and factual knowledge, but structured commonsense knowledge is not captured well. |
| Approach: | They quantitatively investigate the presence of structural commonsense cues in BERT when solving commonsensense tasks and the importance of such cue for the model prediction. |
| Outcome: | The presence of commonsense knowledge is positively correlated to the model accuracy. |
Commonsense Inference in Natural Language Processing (COIN) - Shared Task Report (D19-60)
Copied to clipboard
| Challenge: | The workshop on Commonsense Inference in NLP (COIN) evaluated text understanding systems' ability to draw inferences about facts that are not mentioned in the text, but that are assumed to be common ground. |
| Approach: | They propose to use commonsense knowledge to evaluate systems' ability to answer questions/queries about a text. |
| Outcome: | The proposed tasks evaluated systems in two contexts: Commonsense Inference and Commonsensible Inference. |
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for solving common NLP tasks rely on fine-tuning of pre-trained transformer models. |
| Approach: | They propose a scoring method that casts a plausibility ranking task in full-text format without fine-tuning . they use masked language modeling head tuned during pre-training phase to exploit this method . |
| Outcome: | The proposed method produces strong baselines comparable to supervised approaches. |
Cracking the Contextual Commonsense Code: Understanding Commonsense Reasoning Aptitude of Deep Contextual Representations (D19-60)
Copied to clipboard
| Challenge: | Pretrained deep contextual representations have advanced the state-of-the-art on various commonsense NLP tasks, but we lack a concrete understanding of their capabilities. |
| Approach: | They investigate BERT's ability to encode various commonsense features in its embedding space, but are still deficient in many areas. |
| Outcome: | The proposed model improves performance on a downstream commonsense reasoning task while using minimal data. |
BERTTune: Fine-Tuning Neural Machine Translation with BERTScore (2021.acl-short)
Copied to clipboard
| Challenge: | Neural machine translation models are biased toward limited translation references . BERTScore is a scoring function based on contextual embeddings that overcomes the limitations of n-gram-based metrics. |
| Approach: | They propose to fine-tune models with a new evaluation metric based on contextual embeddings to overcome the limitations of n-gram-based metrics. |
| Outcome: | The proposed training objective improves translations that are different from the translations but close in the contextual embedding space. |
Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning (2020.coling-main)
Copied to clipboard
| Challenge: | Recent work has explored the suitability of pre-trained language models in low resource settings with less than 1,000 training data points. |
| Approach: | They propose to use pool-based active learning to speed up training while keeping the cost of labeling new data constant. |
| Outcome: | The proposed model can be fine-tuned to optimize for low-resource settings while keeping the cost of labeling constant. |
CAUnLP at NLP4IF 2019 Shared Task: Context-Dependent BERT for Sentence-Level Propaganda Detection (D19-50)
Copied to clipboard
| Challenge: | Sentence-level and fragment-level propaganda detection tasks are more challenging compared to document-level detection. |
| Approach: | They propose to use context-dependent input pairs to fine-tune the pretrained propaganda detection BERT to better utilize document information. |
| Outcome: | The proposed system can detect propaganda on document-level, sentence-level and fragment-level. |