Stretching Sentence-pair NLI Models to Reason over Long Documents and Clusters (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent advances in modeling and datasets demonstrate promising performance for NLI. |
| Approach: | They explore the direct zero-shot applicability of NLI models to real applications . they analyze the robustness of models to longer and out-of-domain inputs . |
| Outcome: | The proposed models are robust to longer and out-of-domain inputs and can perform on full documents. |
Similar Papers
Simple but Challenging: Natural Language Inference Models Fail on Simple Sentences (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Natural language inference (NLI) tasks are difficult to perform on large datasets . a small number of simple sentences can improve model performance, authors say . |
| Approach: | They propose to use syntactically simple sentences to test the inference ability of NLI models. |
| Outcome: | The proposed set of simple sentences shows that the models fine-tuned on MNLI and SNLI perform poorly on Simple Pair. |
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on sentence-level inference, which limits its application in downstream NLP problems. |
| Approach: | They propose to construct a large-scale dataset for document-level NLI that can be used to study NLP problems. |
| Outcome: | The proposed model performs well on popular sentence-level benchmarks and generalizes well to out-of-domain NLP tasks that rely on inference at document granularity. |
Deep Learning for Natural Language Inference (N19-5)
Copied to clipboard
| Challenge: | This tutorial discusses cutting-edge research on NLI, including recent advance on dataset development, cutting- edge deep learning models, and highlights from recent research on using NLI to understand capabilities and limits of deep learning for language understanding and reasoning. |
| Approach: | This tutorial discusses cutting-edge research on NLI, including recent advance on dataset development and cutting- edge deep learning models. |
| Outcome: | This tutorial discusses cutting-edge research on NLI, including recent advance on dataset development, cutting- edge deep learning models, and highlights from recent research on using NLI to understand capabilities and limits of deep learning model for language understanding and reasoning. |
A synthetic data approach for domain generalization of NLI models (2024.acl-long)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets are important benchmark tasks for LLMs . however, their realistic performance on out-of-distribution/domain data is less well-understood . a T5-small model trained with our data improves around 7% on average compared to the best alternative dataset . |
| Approach: | They propose a new approach for generating NLI data in diverse domains and lengths . they show that models trained on this data have the best generalization to completely new downstream test settings . |
| Outcome: | The proposed model can be trained on datasets with high-quality examples with meaningful premises and high accuracy. |
Rethinking STS and NLI in Large Language Models (2024.findings-eacl)
Copied to clipboard
| Challenge: | Recent years have seen the rise of large language models (LLMs), where practitioners use task-specific prompts; this was shown to be effective for a variety of tasks. |
| Approach: | They propose to rethink semantic textual similarity (STS) and natural language inference (NLI) models with task-specific prompts and model overconfidence to capture disagreements between human judgements. |
| Outcome: | The proposed models are able to capture human opinions on individual examples without any parameter modifications. |
Can NLI Models Verify QA Systems’ Predictions? (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Recent question answering systems perform well on benchmark datasets, but are not always well-calibrated to spot spurious answers under distribution shifts. |
| Approach: | They propose to use natural language inference to verify whether answers are correct . they leverage large pre-trained models and recent prior datasets to construct powerful question conversion and decontextualization modules. |
| Outcome: | The proposed approach improves the confidence estimation of a QA model across different domains, evaluated in a selective QA setting. |
Breaking NLI Systems with Sentences that Require Simple Lexical Inferences (P18-2)
Copied to clipboard
| Challenge: | a new test set shows the deficiency of state-of-the-art models in inferences that require lexical and world knowledge. |
| Approach: | They create a new NLI test set that shows the deficiency of state-of-the-art models in inferences that require lexical and world knowledge. |
| Outcome: | The new examples are simpler than the SNLI test set, but the state-of-the-art systems perform poorly on it. |
Temporal Reasoning in Natural Language Inference (2020.findings-emnlp)
Copied to clipboard
| Challenge: | We use five new natural language inference (NLI) datasets focused on temporal reasoning. |
| Approach: | They introduce five new natural language inference datasets focused on temporal reasoning. |
| Outcome: | The proposed models capture the temporal reasoning of four existing datasets. |
ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Contract review is a time-consuming procedure that costs companies millions of dollars each year . linguistic characteristics of contracts, such as negations by exceptions, contribute to the difficulty of this task . |
| Approach: | They propose a document-level natural language inference (NLI) task for contracts . they annotate and release the largest corpus to date consisting of 607 annotated contracts a linguistically rich system is proposed . |
| Outcome: | The proposed system is based on a contract review task that includes 607 annotated contracts. |
Enhancing Self-Consistency and Performance of Pre-Trained Language Models through Natural Language Inference (2022.emnlp-main)
Copied to clipboard
Eric Mitchell, Joseph Noh, Siyan Li, Will Armstrong, Ananth Agarwal, Patrick Liu, Chelsea Finn, Christopher Manning
| Challenge: | Existing frameworks for boosting consistency and accuracy of pre-trained NLP models without fine-tuning or re-training are lacking. |
| Approach: | They propose a framework for boosting the consistency and accuracy of pre-trained NLP models using pre-trainer natural language inference models without fine-tuning or re-training. |
| Outcome: | The proposed framework boosts consistency and accuracy of pre-trained NLP models using pre-train natural language inference models without fine-tuning or re-training. |