Development of pre-trained language models for clinical NLP in Spanish (2023.eacl-srw)
Copied to clipboard
| Challenge: | Clinical natural language processing aims to tackle language and prediction tasks using text from medical practice, such as clinical notes, prescriptions, and discharge summaries. |
| Approach: | They propose to build a clinical corpus big enough to implement a functional PLM. |
| Outcome: | The proposed model will be able to handle language and prediction tasks using clinical text while using biomedical and general text. |
Similar Papers
ClinicalT5: A Generative Language Model for Clinical Text (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent generative language models like BART and T5 are gaining popularity with their competitive performance on text generation and tasks cast as generative problems. |
| Approach: | They propose to build domain-specific PLMs through fine-tuning or pre-training from scratch over domain corpora. |
| Outcome: | The proposed model outperforms existing models on domain-specific tasks and compares favorably with its close baselines. |
DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains (2023.acl-long)
Copied to clipboard
Yanis Labrak, Adrien Bazoge, Richard Dufour, Mickael Rouvier, Emmanuel Morin, Béatrice Daille, Pierre-Antoine Gourraud
| Challenge: | Recent studies have shown that pre-trained language models improve performance on a wide range of NLP tasks. |
| Approach: | They propose to use pre-trained language models to train medical domains on French language to compare performance with specialized ones. |
| Outcome: | The proposed models can take advantage of existing biomedical models in a foreign language by further pre-training them on our targeted data. |
Recent Advances in Pre-trained Language Models: Why Do They Work and How Do They Work (2022.aacl-tutorials)
Copied to clipboard
| Challenge: | Pre-trained language models are language models that are pre-taught on large-scaled corpora in a self-supervised fashion. |
| Approach: | This tutorial provides a broad and comprehensive introduction to pre-trained language models . it focuses on emerging methods that enable PLMs to perform diverse downstream tasks . |
| Outcome: | This tutorial focuses on the benefits of pre-trained language models and how to use them in NLP tasks. |
Incorporating medical knowledge in BERT for clinical relation extraction (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (PLMs) are used for diverse NLP tasks such as Information Extraction, Sentiment Analysis and Question/Answering. |
| Approach: | They propose to add medical knowledge to pre-trained language models to facilitate clinical relation extraction using a large text corpus. |
| Outcome: | The proposed model outperforms the state-of-the-art systems on the benchmark i2b2/VA 2010 clinical relation extraction dataset. |
Applications of Natural Language Processing in Clinical Research and Practice (N19-5)
Copied to clipboard
| Challenge: | a tutorial on clinical NLP will introduce students and experts to the field . a focus will be on the use of clinical Nlp in clinical research and practice . |
| Approach: | This tutorial introduces the clinical use of natural language processing (NLP) techniques . it will review techniques and tools developed for the clinical domain . |
| Outcome: | This tutorial will introduce the clinical NLP methodologies and tools at two top universities . the goal of the tutorial is to encourage NLP researchers in the general domain to contribute . |
Comprehensive Study on German Language Models for Clinical and Biomedical Text Understanding (2024.lrec-main)
Copied to clipboard
Ahmad Idrissi-Yaghir, Amin Dada, Henning Schäfer, Kamyar Arzideh, Giulia Baldini, Jan Trienes, Max Hasin, Jeanette Bewersdorff, Cynthia S. Schmidt, Marie Bauer, Kaleb E. Smith, Jiang Bian, Yonghui Wu, Jörg Schlötterer, Torsten Zesch, Peter A. Horn, Christin Seifert, Felix Nensa, Jens Kleesiek, Christoph M. Friedrich
| Challenge: | Pre-trained language models can struggle in specialized domains such as medicine . existing generalpurpose pre-tried models can be used and refined through further pre-training on domainspecific unlabeled data. |
| Approach: | They pre-trained German medical language models on 2.4B tokens from translated public data and 3B token of German clinical data. |
| Outcome: | The proposed models outperform clinical models on various downstream tasks in germany . the authors show that continuous pre-training can match or exceed clinical models trained from scratch . |
Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (2024.findings-acl)
Copied to clipboard
Ran Xu, Hejie Cui, Yue Yu, Xuan Kan, Wenqi Shi, Yuchen Zhuang, May Dongmei Wang, Wei Jin, Joyce Ho, Carl Yang
| Challenge: | Clinical natural language processing (NLP) is a subfield that requires the extraction, analysis, and interpretation of unstructured clinical text. |
| Approach: | They propose a model which infuses knowledge into clinical text generation with LLMs for clinical NLP tasks. |
| Outcome: | The proposed model improves performance across 8 clinical NLP tasks and 18 datasets by 7.7%-8.7% on average. |
MedMT5: An Open-Source Multilingual Text-to-Text LLM for the Medical Domain (2024.lrec-main)
Copied to clipboard
Iker García-Ferrero, Rodrigo Agerri, Aitziber Atutxa Salazar, Elena Cabrio, Iker de la Iglesia, Alberto Lavelli, Bernardo Magnini, Benjamin Molinet, Johana Ramirez-Romero, German Rigau, Jose Maria Villa-Gonzalez, Serena Villata, Andrea Zaninello
| Challenge: | Existing studies on large language models for medical applications have focused on a single language . medical mT5 outperforms both encoders and similar sized text-to-text models in English, French, and Italian benchmarks . |
| Approach: | They propose to train Medical mT5, the first open-source text-to-text multilingual model for the medical domain. |
| Outcome: | The proposed model outperforms encoders and similar sized models on the Spanish, French, and Italian benchmarks while being competitive with current state-of-the-art models in English. |
Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark (2024.emnlp-main)
Copied to clipboard
Fenglin Liu, Zheng Li, Hongjian Zhou, Qingyu Yin, Jingfeng Yang, Xianfeng Tang, Chen Luo, Ming Zeng, Haoming Jiang, Yifan Gao, Priyanka Nigam, Sreyashi Nag, Bing Yin, Yining Hua, Xuan Zhou, Omid Rohanian, Anshul Thakur, Lei Clifton, David Clifton
| Challenge: | Existing studies focus on evaluating large language models in close-ended QA tasks, but many clinical decisions involve answering open-ended questions without pre-set options. |
| Approach: | They construct a benchmark to better understand large language models in the clinic . they use existing datasets to evaluate LLMs in clinical situations . |
| Outcome: | The proposed model outperforms human experts in multiple medical tasks. |
Sequence-to-Sequence Spanish Pre-trained Language Models (2024.lrec-main)
Copied to clipboard
| Challenge: | Spanish language models have demonstrated proficiency in natural language understanding and generation, but there is a scarcity of encoder-decoder models specifically designed for sequence-to-sequence tasks. |
| Approach: | They propose to implement encoder-decoder architectures pre-trained on Spanish corpora . they use them to assess sequence-to-sequence tasks including summarization, question answering . |
| Outcome: | The proposed models outperform models on sequence-to-sequence tasks in Spanish . the models show that they perform well across all tasks, the authors note . |