| Challenge: | Existing approaches to encoding sentences using contextualized encoders are inconsistent . |
| Approach: | They propose to use a cross entropy log-loss objective to improve plausibility . they propose a margin-based loss leads to a more plausible model of plausability . |
| Outcome: | The proposed loss is intuitively wrong when applied to plausibility tasks . the proposed loss leads to a more plausible model of plausability . |
Similar Papers
Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | In the recent past, a popular way of evaluating natural language understanding was to consider a model’s ability to perform natural language inference (NLI) tasks. |
| Approach: | They focus on five different NLI benchmarks across six models of different scales and examine how their accuracies develop during training. |
| Outcome: | The softmax distributions of models align with human label distributions in cases where statements are ambiguous or vague. |
Uncertain Natural Language Inference (2020.acl-main)
Copied to clipboard
| Challenge: | Existing categorical bin assignments for NLI are not effective for entailment inference. |
| Approach: | They propose a refinement of Natural Language Inference that targets direct prediction of subjective probability assessments. |
| Outcome: | The proposed model can be used in pre-training and correlates well with humans. |
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition (2020.acl-main)
Copied to clipboard
| Challenge: | Natural language inference (NLI) is an increasingly important task for natural language understanding . however, the ability of NLI models to make pragmatic inferences remains understudied . |
| Approach: | They use semi-automatically generated sentence pairs to evaluate whether NLI models make pragmatic inferences. |
| Outcome: | The proposed model trains on multiNLI and shows that it learns to draw pragmatic inferences. |
Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for solving common NLP tasks rely on fine-tuning of pre-trained transformer models. |
| Approach: | They propose a scoring method that casts a plausibility ranking task in full-text format without fine-tuning . they use masked language modeling head tuned during pre-training phase to exploit this method . |
| Outcome: | The proposed method produces strong baselines comparable to supervised approaches. |
Language Models can Exploit Cross-Task In-context Learning for Data-Scarce Novel Tasks (2024.acl-long)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have transformed NLP with their remarkable In-context Learning capabilities. |
| Approach: | They propose to use large language models to generalize from labeled examples of predefined tasks to novel tasks . they use biological neurons and the Transformer architecture to study the potential for information sharing across tasks. |
| Outcome: | The proposed model can generalize from labeled examples of predefined tasks to novel tasks despite no examples from the target task in the context. |
Embarrassingly Simple Performance Prediction for Abductive Natural Language Inference (2022.naacl-main)
Copied to clipboard
| Challenge: | a method for learning an NLI model is time-consuming and resource-intensive, but it can save time and resources. |
| Approach: | They propose a method for predicting model performance without fine-tuning it . they compare sentence embeddings with cosine similarity to classifiers . |
| Outcome: | The proposed method can save time and resources by comparing pre-trained models to real-world datasets. |
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning. |
| Approach: | They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI. |
| Outcome: | The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks. |
Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current models learn from annotation artefacts and dataset biases, but it is unclear to what extent they are learning the task of NLI. |
| Approach: | They propose a logical reasoning framework that allows models to learn from annotation artefacts and dataset biases. |
| Outcome: | The proposed model outperforms humans on in-distribution test sets without using span labels . the model is more robust in a reduced data setting, and out-of-disturbance performance is improved . |
Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference (P19-1)
Copied to clipboard
| Challenge: | lexical overlap heuristics are effective for frequent example types but break down in more challenging cases. |
| Approach: | They propose to use a set of examples to test whether a sentence entails another . they propose to adopt three fallible syntactic heuristics for statistical NLI models . |
| Outcome: | The proposed model performs poorly on the HANS dataset, suggesting they have adopted these heuristics. |
Leveraging Entailment Judgements in Cross-Lingual Summarisation (2024.findings-acl)
Copied to clipboard
| Challenge: | Synthetically created cross-lingual summarisation datasets are prone to include document-summary pairs where the reference summary is unfaithful to the corresponding document. |
| Approach: | They propose to use off-the-shelf cross-lingual Natural Language Inference to evaluate faithfulness of reference and model generated summaries and use unlikelihood loss to teach a model about unfaithful summary sequences. |
| Outcome: | The proposed approach evaluates faithfulness of reference and model generated summaries and uses unlikelihood loss to teach a model about unfaithful summary sequences. |