| Challenge: | State of the art models with deep neural networks lack generalization capabilities in specialized domains where training data is limited. |
| Approach: | They propose a dataset annotated by doctors performing a natural language inference task grounded in the medical history of patients. |
| Outcome: | The proposed model outperforms existing models in the clinical domain by incorporating domain knowledge from external data and lexical sources. |
Similar Papers
MedNLI Is Not Immune: Natural Language Inference Artifacts in the Clinical Domain (2021.acl-short)
Copied to clipboard
| Challenge: | a large number of crowdworker-constructed datasets have been used to conduct natural language inference (NLI) on unstructured, domainspecific texts such as patient notes, pathology reports, and scientific papers. |
| Approach: | They investigate whether MedNLI contains lexical and syntactic annotation artifacts associated with annotation process that allow hypothesis-only classifiers to achieve better-than-random performance. |
| Outcome: | The proposed model outperforms a majority-class baseline model on a physician-annotated dataset with premises extracted from clinical notes. |
A synthetic data approach for domain generalization of NLI models (2024.acl-long)
Copied to clipboard
| Challenge: | Natural Language Inference (NLI) datasets are important benchmark tasks for LLMs . however, their realistic performance on out-of-distribution/domain data is less well-understood . a T5-small model trained with our data improves around 7% on average compared to the best alternative dataset . |
| Approach: | They propose a new approach for generating NLI data in diverse domains and lengths . they show that models trained on this data have the best generalization to completely new downstream test settings . |
| Outcome: | The proposed model can be trained on datasets with high-quality examples with meaningful premises and high accuracy. |
NLI4CT: Multi-Evidence Natural Language Inference for Clinical Trial Reports (2023.emnlp-main)
Copied to clipboard
| Challenge: | Clinical trial reports (CTRs) are indispensable for the development of personalized medicine. |
| Approach: | They propose a resource to help researchers interpret clinical trial reports . they use natural language inference to compute textual entailment . |
| Outcome: | The proposed resource is the first to cover interpretation of full clinical trial reports . it includes tasks to determine inference relation between natural language statements and CTRs . |
Neural Natural Language Inference Models Enhanced with External Knowledge (P18-1)
Copied to clipboard
| Challenge: | Existing datasets that allow for complex models to be trained are limited . if data is not available, can machines learn all knowledge needed to perform natural language inference? |
| Approach: | They propose to enrich neural natural language inference models with external knowledge . they propose to use this knowledge to build NLI models to leverage it . |
| Outcome: | The proposed models improve on the SNLI and MultiNLI datasets. |
Incorporating Domain Knowledge into Medical NLI using Knowledge Graphs (D19-1)
Copied to clipboard
| Challenge: | Recent studies have shown that structured domain knowledge can be used for textual inference tasks in the medical domain. |
| Approach: | They propose to integrate structured domain knowledge into a knowledge graph for the Medical NLI task. |
| Outcome: | The proposed approach improves the baseline BioELMo architecture for the Medical NLI task. |
BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Biomedical research has progressed at a tremendous pace, with PubMed2 indexing well over 1M publications per year in the past eight years. |
| Approach: | They propose a semi-supervised procedure that bootstraps biomedical NLI datasets from positive entailment examples present in biomedically published texts. |
| Outcome: | The proposed procedure bootstraps biomedical NLI datasets from positive entailment examples from biomedically challenging texts. |
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on sentence-level inference, which limits its application in downstream NLP problems. |
| Approach: | They propose to construct a large-scale dataset for document-level NLI that can be used to study NLP problems. |
| Outcome: | The proposed model performs well on popular sentence-level benchmarks and generalizes well to out-of-domain NLP tasks that rely on inference at document granularity. |
Deep Learning for Natural Language Inference (N19-5)
Copied to clipboard
| Challenge: | This tutorial discusses cutting-edge research on NLI, including recent advance on dataset development, cutting- edge deep learning models, and highlights from recent research on using NLI to understand capabilities and limits of deep learning for language understanding and reasoning. |
| Approach: | This tutorial discusses cutting-edge research on NLI, including recent advance on dataset development and cutting- edge deep learning models. |
| Outcome: | This tutorial discusses cutting-edge research on NLI, including recent advance on dataset development, cutting- edge deep learning models, and highlights from recent research on using NLI to understand capabilities and limits of deep learning model for language understanding and reasoning. |
Probing Pre-Trained Language Models for Disease Knowledge (2021.findings-acl)
Copied to clipboard
| Challenge: | Pre-trained language models perform medical reasoning tasks, but standard benchmarks lack examples that require such forms of reasoning. |
| Approach: | They propose a medical reasoning benchmark that uses pre-trained language models to analyze medical reasoning in the biomedical domain. |
| Outcome: | The proposed benchmarks are based on pre-trained language models that perform medical reasoning tasks. |
Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding (2024.lrec-main)
Copied to clipboard
Ahmad Idrissi-Yaghir, Amin Dada, Henning Schäfer, Kamyar Arzideh, Giulia Baldini, Jan Trienes, Max Hasin, Jeanette Bewersdorff, Cynthia S. Schmidt, Marie Bauer, Kaleb E. Smith, Jiang Bian, Yonghui Wu, Jörg Schlötterer, Torsten Zesch, Peter A. Horn, Christin Seifert, Felix Nensa, Jens Kleesiek, Christoph M. Friedrich
| Challenge: | Pre-trained language models can struggle in specialized domains such as medicine . existing generalpurpose pre-tried models can be used and refined through further pre-training on domainspecific unlabeled data. |
| Approach: | They pre-trained German medical language models on 2.4B tokens from translated public data and 3B token of German clinical data. |
| Outcome: | The proposed models outperform clinical models on various downstream tasks in germany . the authors show that continuous pre-training can match or exceed clinical models trained from scratch . |