Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference (2022.emnlp-main)
Copied to clipboard
Eric Mitchell, Joseph Noh, Siyan Li, Will Armstrong, Ananth Agarwal, Patrick Liu, Chelsea Finn, Christopher Manning
| Challenge: | Existing frameworks for boosting consistency and accuracy of pre-trained NLP models without fine-tuning or re-training are lacking. |
| Approach: | They propose a framework for boosting the consistency and accuracy of pre-trained NLP models using pre-trainer natural language inference models without fine-tuning or re-training. |
| Outcome: | The proposed framework boosts consistency and accuracy of pre-trained NLP models using pre-train natural language inference models without fine-tuning or re-training. |
Similar Papers
Can NLI Models Verify QA Systems’ Predictions? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent question answering systems perform well on benchmark datasets, but are not always well-calibrated to spot spurious answers under distribution shifts. |
| Approach: | They propose to use natural language inference to verify whether answers are correct . they leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules. |
| Outcome: | The proposed approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. |
Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in modeling and datasets demonstrate promising performance for NLI. |
| Approach: | They explore the direct zero-shot applicability of NLI models to real applications . they analyze the robustness of models to longer and out-of-domain inputs . |
| Outcome: | The proposed models are robust to longer and out-of-domain inputs and can perform on full documents. |
Simple but Challenging: Natural Language Inference Models Fail on Simple Sentences (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Natural language inference (NLI) tasks are difficult to perform on large datasets . a small number of simple sentences can improve model performance, authors say . |
| Approach: | They propose to use syntactically simple sentences to test the inference ability of NLI models. |
| Outcome: | The proposed set of simple sentences shows that the models fine-tuned on MNLI and SNLI perform poorly on Simple Pair. |
Measuring and Improving Consistency in Pretrained Language Models (2021.tacl-1)
Copied to clipboard
Yanai Elazar, Nora Kassner, Shauli Ravfogel, Abhilasha Ravichander, Eduard Hovy, Hinrich Schütze, Yoav Goldberg
| Challenge: | In this paper, we examine whether pretrained language models are consistent with factual knowledge. |
| Approach: | They propose a method to improve consistency of pretrained language models . consistency is a desirable property of a good language understanding model, they argue . |
| Outcome: | The proposed model improves consistency and shows that it is effective. |
Complex Reasoning in Natural Language (2023.acl-tutorials)
Copied to clipboard
| Challenge: | Recent research shows that pretrained language models are often brittle for complex reasoning tasks. |
| Approach: | They propose to use pre-trained language models to teach machines to reason over texts . they will review recent promising approaches to tackling complex reasoning tasks . |
| Outcome: | This tutorial reviews promising approaches to complex reasoning tasks . it reviews the methods that can be used to augment models with robustness . |
SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization (2020.acl-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning pre-trained models fail to generalize to unseen data. |
| Approach: | They propose a framework for robust and efficient fine-tuning for pre-trained models . proposed framework achieves new state-of-the-art performance on a number of NLP tasks . |
| Outcome: | The proposed framework outperforms the state-of-the-art T5 model on GLUE, SNLI, SciTail and ANLI. |
Coherence boosting: When your pretrained language model is not paying enough attention (2022.acl-long)
Copied to clipboard
| Challenge: | Long-range semantic coherence remains a challenge in automatic language generation and understanding. |
| Approach: | They propose a procedure that increases a model’s focus on a long context by distributional analyses of generated ordinary text and dialog responses. |
| Outcome: | The proposed procedure increases the model's focus on a long context. |
Logical Reasoning with Span-Level Predictions for Interpretable and Robust NLI Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Current models learn from annotation artefacts and dataset biases, but it is unclear to what extent they are learning the task of NLI. |
| Approach: | They propose a logical reasoning framework that allows models to learn from annotation artefacts and dataset biases. |
| Outcome: | The proposed model outperforms humans on in-distribution test sets without using span labels . the model is more robust in a reduced data setting, and out-of-disturbance performance is improved . |
Benchmarking Query-Conditioned Natural Language Inference (2025.findings-acl)
Copied to clipboard
| Challenge: | Query-conditioned natural language inference (QC-NLI) is a new approach to detect inconsistencies in large language models. |
| Approach: | They propose a task of Query-Conditioned Natural Language Inference to determine the semantic relationship between two documents conditioned on a query. |
| Outcome: | The proposed task is based on a query-conditioned natural language inference (QC-NLI) it is used to determine the relationship between the premise and hypothesis given a given query. |
Embarrassingly Simple Performance Prediction for Abductive Natural Language Inference (2022.naacl-main)
Copied to clipboard
| Challenge: | a method for learning an NLI model is time-consuming and resource-intensive, but it can save time and resources. |
| Approach: | They propose a method for predicting model performance without fine-tuning it . they compare sentence embeddings with cosine similarity to classifiers . |
| Outcome: | The proposed method can save time and resources by comparing pre-trained models to real-world datasets. |