AdvEntuRe: Adversarial Training for Textual Entailment with Knowledge-Guided Examples (P18-1)
Copied to clipboard
| Challenge: | Recent deep learning entailment systems have achieved close to human level performance on large datasets, but the problem is far from solved. |
| Approach: | They propose a knowledge-guided adversarial example generator for incorporating large lexical resources into entailment models via only a handful of rule templates and a natural language example generator that iteratively adjusts to the discriminator’s weaknesses. |
| Outcome: | The proposed methods increase accuracy by 4.7% on SciTail and 2.8% on a 1% sub-sample of SNLI. |
Similar Papers
New Protocols and Negative Results for Textual Entailment Data Collection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language inference data has proven useful in benchmarking and as pretraining data for tasks requiring language understanding. |
| Approach: | They propose four alternative protocols to improve annotation quality and diversity . they use 8.5k-example training sets to compare different protocols . |
| Outcome: | The proposed protocols improve the ease of training and quality of the examples. |
Embedding WordNet Knowledge for Textual Entailment (C18-1)
Copied to clipboard
| Challenge: | Existing deep learning models for textual entailment do not require any feature engineering or linguistic analysis. |
| Approach: | They propose to embed WordNet-derived lexical entailment relations into specially-learned word vectors and incorporate them into a decomposable attention model for textual enlightment. |
| Outcome: | The proposed model significantly improves on the SICK and SNLI datasets. |
Adversarial NLI: A New Benchmark for Natural Language Understanding (2020.acl-main)
Copied to clipboard
| Challenge: | a new large-scale NLI benchmark dataset is presented to test models on a variety of popular NLIs. |
| Approach: | They propose a large-scale NLI benchmark dataset that is iteratively compared with a human-and-model-in-the-loop procedure. |
| Outcome: | The proposed method can be applied in a never-ending learning scenario, becoming a moving target for NLU, rather than a static benchmark that will quickly saturate. |
BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Biomedical research has progressed at a tremendous pace, with PubMed2 indexing well over 1M publications per year in the past eight years. |
| Approach: | They propose a semi-supervised procedure that bootstraps biomedical NLI datasets from positive entailment examples present in biomedically published texts. |
| Outcome: | The proposed procedure bootstraps biomedical NLI datasets from positive entailment examples from biomedically challenging texts. |
Generating Natural Language Adversarial Examples (D18-1)
Copied to clipboard
| Challenge: | Recent research has shown that deep neural networks are vulnerable to adversarial examples, perturbations to correctly classified examples which can cause the model to misclassify. |
| Approach: | They propose to generate adversarial examples that fool well-trained sentiment analysis and textual entailment models by using a black-box population-based optimization algorithm. |
| Outcome: | The proposed model is able to fool well-trained sentiment analysis and textual entailment models with success rates of 97% and 70%, respectively. |
GAN-BERT: Generative Adversarial Learning for Robust Text Classification with a Bunch of Labeled Examples (2020.acl-main)
Copied to clipboard
| Challenge: | Recent Transformer-based architectures provide impressive results in many NLP tasks, but obtaining high-quality annotated data is expensive and time consuming. |
| Approach: | They propose a semisupervised learning method that ex- tends the fine-tuning of BERT-like architectures with unlabeled data in a generative adversarial setting. |
| Outcome: | The proposed method reduces the requirement for annotated examples while achieving good performance in sentence classification tasks. |
A Closer Look into the Robustness of Neural Dependency Parsers Using Better Adversarial Examples (2021.findings-acl)
Copied to clipboard
| Challenge: | Neural network-based models have been successful in a wide range of NLP tasks, but their performance is undermined by adversarial examples that would pose no confusion for humans. |
| Approach: | They propose a method to generate high-quality adversarial examples with a higher number of candidate generators and stricter filters and then verify their quality using automatic and human evaluations. |
| Outcome: | The proposed method improves the robustness of English parsing models by relying on adversarial training and model ensembling. |
KBGAN: Adversarial Learning for Knowledge Graph Embeddings (N18-1)
Copied to clipboard
| Challenge: | Existing knowledge graph embedding techniques lack the capability to access similarities between entities and relations. |
| Approach: | They propose an adversarial learning framework to improve knowledge graph embedding models . they use one knowledge graph embedded model as a negative sample generator . |
| Outcome: | The proposed framework improves the performance of knowledge graph embedding models on a link prediction task. |
Bridging Knowledge Gaps in Neural Entailment via Symbolic Models (D18-1)
Copied to clipboard
| Challenge: | Textual entailment models focus on lexical gaps but rarely on knowledge gaps. |
| Approach: | They propose a fact-level decomposition of the hypothesis and a knowledge lookup module to fill knowledge gaps in Science Entailment task. |
| Outcome: | The proposed model outperforms the base model on the SciTail dataset by 3% and 5% on the textual premise and the structured knowledge base. |
Asking Crowdworkers to Write Entailment Examples: The Best of Bad Options (2020.aacl-main)
Copied to clipboard
| Challenge: | Large-scale natural language inference datasets are available for non-expert crowdsourcing. |
| Approach: | They propose to use crowdworker writing to create candidate pairs for annotation . they find crowdworkers writing is the best known protocol for entailment data . |
| Outcome: | The proposed protocol outperforms the baseline protocol in evaluations of generalization and transfer learning. |