PM2F2N: Patient Multi-view Multi-modal Feature Fusion Networks for Clinical Outcome Prediction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods focused on time series data but ignored clinical notes . fusion of multi-modal features of patients from different views is not feasible due to the time series and clinical notes data being stored as time series. |
| Approach: | They propose to combine time series and clinical notes to fuse multi-modal features of patients from different perspectives using graph neural networks. |
| Outcome: | The proposed method is superior to existing models on MIMIC-III benchmark. |
Similar Papers
How to leverage the multimodal EHR data for better medical prediction? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Using deep learning to improve healthcare is challenging due to the complexity of EHR data. |
| Approach: | They propose a method to integrate clinical notes from EHR and combine them with different data to improve prediction performance. |
| Outcome: | The proposed model outperforms the state-of-the-art method without clinical notes on two prediction tasks. |
Data Drift in Clinical Outcome Prediction from Admission Notes (2024.lrec-main)
Copied to clipboard
Paul Grundmann, Jens-Michalis Papaioannou, Tom Oberhauser, Thomas Steffek, Amy Siu, Wolfgang Nejdl, Alexander Loeser
| Challenge: | a pivotal dataset for clinical NLP research was released in 2016 . public access to such datasets is limited due to privacy and ethical concerns . |
| Approach: | They propose a novel clinical outcome prediction dataset based on MIMIC-IV . they provide initial insights into the performance of models trained on MIDIC-III . |
| Outcome: | The proposed dataset aims to probe the robustness and generalization of clinical outcome prediction models . the study focuses on challenges tied to evolving documentation standards and changing codes in the ICD taxonomy . |
LIFTED: Multimodal Clinical Trial Outcome Prediction via Large Language Models and Mixture-of-Experts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Clinical trials are costly and pivotal processes that require substantial expenses . a new approach to integrate multimodal data for clinical outcome prediction is needed . |
| Approach: | a proposed framework transforms modality-specific data into natural language descriptions . a sparse Mixture-of-Experts mechanism then identifies shared patterns across modalities . |
| Outcome: | a proposed framework outperforms baseline methods in predicting clinical trial outcomes . it transforms modality-specific data into natural language descriptions, encoded via unified encoders . |
Patient Outcome and Zero-shot Diagnosis Prediction with Hypernetwork-guided Multitask Learning (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances apply artificial intelligence to predict clinical events or infer the probable diagnosis for clinical decision support. |
| Approach: | They propose a hypernetwork-based approach that generates task-conditioned parameters and coefficients of multitask prediction heads to learn task-specific prediction and balance the multitask learning. |
| Outcome: | Experiments on clinical notes from the real-world MIMIC database show that the proposed model can achieve better performance than baselines and improve zero-shot prediction on unseen diagnoses. |
Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration (2021.eacl-main)
Copied to clipboard
Betty van Aken, Jens-Michalis Papaioannou, Manuel Mayrdorfer, Klemens Budde, Felix Gers, Alexander Loeser
| Challenge: | Clinical decision support systems can help in situations where the patient's development is predicted based on textual data. |
| Approach: | They propose to use clinical outcome pre-training to integrate knowledge about patient outcomes from multiple public sources into the models. |
| Outcome: | The proposed model improves performance against several baselines and demonstrates that it is transferable and can be used in clinical decision support systems. |
Literature-Augmented Clinical Outcome Prediction (2022.findings-naacl)
Copied to clipboard
| Challenge: | Existing approaches to clinical outcome prediction use only clinical notes and general biomedical literature. |
| Approach: | They propose to retrieve patient-specific medical literature and incorporate it into predictive models by combining clinical notes with language models. |
| Outcome: | The proposed approach boosts predictive performance on three important clinical tasks in comparison to strong LM baselines, increasing F1 by up to 5 points and precision@Top-K by a large margin of over 25%. |
Seeing Is Believing! towards Knowledge-Infused Multi-modal Medical Dialogue Generation (2024.lrec-main)
Copied to clipboard
Abhisek Tiwari, Shreyangshu Bera, Preeti Verma, Jaithra Varma Manthena, Sriparna Saha, Pushpak Bhattacharyya, Minakshi Dhar, Sarbajeet Tiwari
| Challenge: | Existing models of disease diagnosis using AI do not use knowledge infusion. |
| Approach: | They propose a transformer-based, knowledge-infused multi-modal medical dialogue generation framework . they propose 'discourse-aware' image identifier that recognizes signs and their severity . |
| Outcome: | The proposed model outperforms state-of-the-art models by 7.84% in the english language. |
Attention Networks for Augmenting Clinical Text with Support Sets for Diagnosis Prediction (2022.coling-1)
Copied to clipboard
| Challenge: | Clinical language models may suffer from imbalanced vocabulary for describing diseases or symptoms. |
| Approach: | They propose to augment clinical text with potentially complementary diagnostic codes from prior admissions or as they emerge during differential diagnosis to improve the performance. |
| Outcome: | The proposed approach outperforms the previous state-of-the-art PubMedBERT by up 3% points. |
Causal Representation Learning from Multimodal Clinical Records under Non-Random Modality Missingness (2025.emnlp-main)
Copied to clipboard
| Challenge: | Clinical notes are often missing from clinical notes, resulting in modality missing-not-at-random (MMNAR) . large language models fine-tuned or adapted to clinical tasks have shown promise in medical reasoning, outcome prediction, and decision support. |
| Approach: | They propose a framework that leverages observed data and informative missingness in multimodal clinical records. |
| Outcome: | The proposed framework leverages observed data and informative missingness in multimodal clinical records. |
Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learning (2023.acl-long)
Copied to clipboard
| Challenge: | Clinical notes of patient EHRs contain valuable information from healthcare professionals, but have been underutilized due to their difficult-to-understand contents and complex hierarchies. |
| Approach: | They propose to use clinical notes to learn more balanced knowledge from EHRs by assembling useful neutral words with rare keywords via note and taxonomy level hyperedges. |
| Outcome: | The proposed method can retain clinical semantic information by (1) frequent neutral words and (2) hierarchies with imbalanced distribution. |