Papers by Ayman Ali
MedRedFlag: Investigating how LLMs Redirect Misconceptions in Real-World Health Communication (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly being used by lay users for medical advice, but they have not yet been tested for this crucial competency. |
| Approach: | They develop a semi-automated pipeline to curate MedRedFlag, a dataset of 1100+ reddit questions that require redirection. |
| Outcome: | The proposed pipeline compares state-of-the-art LLMs to those from clinicians to find out how they perform under real-world health communication. |
“What’s Up, Doc?”: Analyzing How Users Seek Health Information in Large-Scale Conversational AI Datasets (2025.findings-emnlp)
Copied to clipboard
Akshay Paruchuri, Maryam Aziz, Rohit Vartak, Ayman Ali, Best Uchehara, Xin Liu, Ishan Chatterjee, Monica Agrawal
| Challenge: | a growing number of people are seeking healthcare information from large language models via chatbots, yet the nature and inherent risks of these interactions remain unexplored. |
| Approach: | They use a curated dataset of 11K real-world conversations composed of 25K user messages to analyze user interactions across 21 health specialties. |
| Outcome: | The proposed dataset consists of 11K real-world conversations composed of 25K user messages. |