Self-Supervised Intermediate Fine-Tuning of Biomedical Language Models for Interpreting Patient Case Descriptions (2022.coling-1)
Copied to clipboard
| Challenge: | Existing work has found that biomedical language models lack the knowledge needed for such tasks. |
| Approach: | They propose to fine-tune biomedical language models on the task of predicting masked medical concepts from PubMed abstracts to improve their performance. |
| Outcome: | The proposed strategy improves the performance of biomedical language models on the task of predicting masked medical concepts from patient case descriptions. |
Similar Papers
LlamaCare: An Instruction Fine-Tuned Large Language Model for Clinical NLP (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models have shown remarkable abilities in generating natural texts . applying LLMs to clinical domain still poses significant challenges . |
| Approach: | They propose a method of instruction fine-tuning for adapting large language models to clinical domains . they generate instructions, inputs, and outputs covering a wide spectrum of clinical services . |
| Outcome: | The proposed method outperforms baseline LLMs on clinical tasks . it requires domain adaptation, task-specific learning, and reliability . |
Improving Pre-trained Language Model Sensitivity via Mask Specific losses: A case study on Biomedical NER (2024.naacl-long)
Copied to clipboard
| Challenge: | Fine-tuning is the prevailing practice for adapting language models (LMs) to new domains. |
| Approach: | They propose a mask specific language model that weights the importance of domain-specific terms during fine-tuning to avoid insensitivity. |
| Outcome: | The proposed approach outperforms advanced masking strategies such as span- and PMI-based masking. |
Boosting Low-Resource Biomedical QA via Entity-Aware Masking Strategies (2021.eacl-main)
Copied to clipboard
| Challenge: | Biomedical question-answering (QA) provides users with high-quality information from a vast scientific literature. |
| Approach: | They propose to use a biomedical entity-aware masking strategy to fine-tune masked language models to their domains. |
| Outcome: | The proposed approach is an adaptation process for masked LMs, not memory or components. |
Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach (2021.naacl-main)
Copied to clipboard
| Challenge: | Fine-tuned pre-trained language models (LMs) have enormous success in many natural language processing tasks, but they still require excessive labeled data in the fine-tuning stage. |
| Approach: | They propose a framework to enable fine-tuning pre-trained language models with weak supervision without any labeled data. |
| Outcome: | The proposed framework outperforms the strongest baseline and achieves competitive performance with fully-supervised fine-tuning methods. |
BioReddit: Word Embeddings for User-Generated Biomedical NLP (D19-62)
Copied to clipboard
| Challenge: | a corpus of medical-themed posts was scrapped from Reddit to train word embeddings on downstream tasks. |
| Approach: | They propose to train word embeddings from a corpus of medical forums from reddit scrapping posts from medical-themed subreddits. |
| Outcome: | The proposed system outperforms embeddings trained on general purpose data or on scientific papers when applied on user-generated content. |
Boosting Transformers and Language Models for Clinical Prediction in Immunotherapy (2023.acl-industry)
Copied to clipboard
| Challenge: | Current machine learning approaches to predict clinical outcomes are limited to tabular data and are not applicable to clinical prediction. |
| Approach: | They investigate the potential of transformers to improve clinical prediction compared to conventional machine learning approaches and address the challenge of few-shot learning in predicting rare disease areas. |
| Outcome: | The proposed model improves the accuracy of baseline models and language models under few-shot regimes and shows that it is more accurate than previous models. |
Incorporating medical knowledge in BERT for clinical relation extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are used for diverse NLP tasks such as Information Extraction, Sentiment Analysis and Question/Answering. |
| Approach: | They propose to add medical knowledge to pre-trained language models to facilitate clinical relation extraction using a large text corpus. |
| Outcome: | The proposed model outperforms the state-of-the-art systems on the benchmark i2b2/VA 2010 clinical relation extraction dataset. |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
Generative Biomedical Entity Linking via Knowledge Base-Guided Pre-training and Synonyms-Aware Fine-tuning (2022.naacl-main)
Copied to clipboard
| Challenge: | Generative methods for biomedical entity linking (EL) use synonyms knowledge from knowledge bases (KB) this is not trivial to inject into a generative method, but it is cost-effective. |
| Approach: | They propose to inject synonyms knowledge into a generative model of biomedical EL by constructing synthetic samples with synonyms and definitions from KB and requiring the model to recover concept names. |
| Outcome: | The proposed method achieves state-of-the-art results on several biomedical EL tasks without candidate selection. |
Can Synthetic Text Help Clinical Named Entity Recognition? A Study of Electronic Health Records in French (2023.eacl-main)
Copied to clipboard
| Challenge: | In sensitive domains, the sharing of corpora is restricted due to confidentiality, copyrights or trade secrets. |
| Approach: | They use auto-regressive neural models to generate a clinical case corpus annotated with clinical entities and evaluate it for a named entity recognition task. |
| Outcome: | The proposed model can produce clinical case corpus annotated with clinical entities while maintaining confidentiality. |