ProMedTS: A Self-Supervised, Prompt-Guided Multimodal Approach for Integrating Medical Text and Time Series (2025.findings-acl)
Copied to clipboard
Shuai Niu, Jing Ma, Hongzhan Lin, Liang Bai, Zhihua Wang, V. W., Richard Yi Da Xu, Guo Li, Xian Yang
| Challenge: | Large language models excel at processing unstructured data, but integrating time series data with text remains a challenge. |
| Approach: | They propose a self-supervised multimodal framework that uses prompt-guided learning to unify heterogeneous data types. |
| Outcome: | The proposed framework outperforms state-of-the-art approaches on disease diagnosis tasks using real-world datasets. |
Similar Papers
RePrompT: Recurrent Prompt Tuning for Integrating Structured EHR Encoders with Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have shown promising results for mining EHRs . translating time-stamped sequences into plain text can obscure both temporal structure and code identities, weakening the ability to capture code co-occurrence and longitudinal regularities. |
| Approach: | They propose a time-aware LLM framework that integrates structured EHR encoders through prompt tuning without modifying underlying architectures. |
| Outcome: | Experiments on MIMIC-III and MIMIC IV show that RePrompT outperforms both EHR-based and LLM-based baselines across multiple clinical prediction tasks. |
LENS: LLM-Enabled Narrative Synthesis for Mental Health by Aligning Multimodal Sensing with Language Models (2026.acl-long)
Copied to clipboard
Wenxuan Xu, Arvind Pillai, Subigya Nepal, Amanda C. Collins, Daniel M Mackin, Michael V. Heinz, Tess Z Griffin, Nicholas C. Jacobson, Andrew Campbell
| Challenge: | Current LLMs cannot natively ingest long-duration sensor streams and paired sensor–text datasets are scarce. |
| Approach: | They propose a framework that aligns multimodal sensing data with language models to generate clinically grounded mental-health narratives. |
| Outcome: | The proposed framework outperforms baselines on NLP metrics and task-specific measures of symptom severity and clinically meaningful narratives. |
PromptEHR: Conditional Electronic Healthcare Records Generation with Prompt Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for generating longitudinal multimodal EHRs are limited due to privacy concerns. |
| Approach: | They propose to generate longitudinal multimodal EHRs by unconditional generation or longitudinal inference . existing methods generate single-modal E HRs by conditional generation or by longitudinal inferment . |
| Outcome: | The proposed method is more flexible and controllable than existing methods and is more cost-effective than existing ones. |
Time-RA: Towards Time Series Reasoning for Anomaly Diagnosis with LLM Feedback (2026.findings-acl)
Copied to clipboard
Yiyuan Yang, Zichuan Liu, Lei Song, Kai Ying, Stephen Wang, Joshua Thomas Bamford, Svitlana Vyetrenko, Jiang Bian, Qingsong Wen
| Challenge: | Time series anomaly detection (TSAD) has traditionally focused on binary classification and lacks the fine-grained categorization and explanatory reasoning required for transparent decision-making. |
| Approach: | They propose a time-series reasoning task that reformulates TSAD from discriminative to reasoning-intensive paradigm. |
| Outcome: | The proposed task reformulates TSAD from discriminative to reasoning-intensive paradigm. |
Guiding Medical Vision-Language Models with Diverse Visual Prompts: Framework Design and Comprehensive Exploration of Prompt Variations (2025.naacl-long)
Copied to clipboard
| Challenge: | Current vision-language models lack the ability to focus on specific areas designated by humans . a new framework that integrates medical entity extraction, visual prompt generation, and dataset adaptation is proposed to improve visual prompt-guided fine-tuning. |
| Approach: | They propose to use visual prompts to guide and enhance formation of region-specific attention. |
| Outcome: | The proposed framework outperforms state-of-the-art large vision-language models on medical datasets. |
MultiQT: Multimodal learning for real-time question tracking in speech (2020.acl-main)
Copied to clipboard
Jakob D. Havtorn, Jan Latko, Joakim Edin, Lars Maaløe, Lasse Borgholt, Lorenzo Belgrano, Nicolai Jacobsen, Regitze Sdun, Željko Agić
| Challenge: | a novel multimodal approach to real-time sequence labeling in speech is proposed . the model treats speech and its own textual representation as two separate modalities . |
| Approach: | They propose a multimodal approach to real-time sequence labeling in speech . they use audio and transcription to jointly learn from a phone call . results show similar pattern of improvements with multimodal learning . |
| Outcome: | The proposed model shows significant gains under adverse noise and limited training data compared to text or audio only under adverse conditions and generalizes to medical symptoms detection. |
Multimodal Dual-Path Decoding for Medical Report Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Current methods for radiology report generation rely on encoder-decoder based frameworks that fail to integrate multimodal clinical evidence with domain-specific knowledge. |
| Approach: | They propose a multimodal dual-path framework that synergistically integrates large vision-language models and large language models for radiology report generation. |
| Outcome: | The proposed framework improves on the public MIMIC-CXR benchmark and shows that it is superior to state-of-the-art models. |
Large Language Models with Temporal Reasoning for Longitudinal Clinical Summarization and Prediction (2025.findings-emnlp)
Copied to clipboard
Maya Kruse, Shiyue Hu, Nicholas Derby, Yifu Wu, Samantha Stonbraker, Bingsheng Yao, Dakuo Wang, Elizabeth M. Goldberg, Yanjun Gao
| Challenge: | Recent advances in large language models have shown potential in clinical text summarization, but their ability to handle long patient trajectories with multi-modal data spread across time remains underexplored. |
| Approach: | They evaluate open-source large language models, their Retrieval Augmented Generation variants and chain-of-thought prompting on long-context clinical summarization and prediction. |
| Outcome: | The proposed models can synthesize structured and unstructured EHR data while reasoning over temporal coherence. |
Augur: Modeling Covariate Causal Associations in Time Series via Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) have emerged as a promising avenue for time series forecasting . existing approaches face limitations such as marginalized role in model architectures and lack of interpretability. |
| Approach: | They propose a framework that exploits LLM causal reasoning to discover and use directed causal associations among covariates. |
| Outcome: | The proposed model improves predictive accuracy while yielding transparent, traceable reasoning about variable interactions. |
LLaTiSA: Towards Difficulty-Stratified Time Series Reasoning from Visual Perception to Semantics (2026.findings-acl)
Copied to clipboard
| Challenge: | Current research hinders the development of unified Time Series Reasoning Models (TSRMs) time series data are a fundamental modality for capturing the temporal dynamics of complex systems. |
| Approach: | They propose a time series reasoning model that integrates visualized patterns with precision-calibrated numerical tables to enhance the temporal perception of Vision-Language Models. |
| Outcome: | The proposed model outperforms existing models and exhibits robust out-of-distribution generalization across diverse tasks and real-world scenarios. |