Papers by Mohita Chowdhury
ASTRID - An Automated and Scalable TRIaD for the Evaluation of RAG-based Clinical Question Answering Systems (2025.findings-acl)
Copied to clipboard
| Challenge: | Current automated RAG metrics perform poorly in clinical and conversational use cases. |
| Approach: | They propose an automated and scaleable TRIaD for evaluating clinical QA systems leveraging Retrieval Augmented Generation (RAG) metric consisting of three metrics: Context Relevance (CR), Refusal Accuracy (RA), and Conversational Faithfulness (CF). |
| Outcome: | The proposed metric captures the faithfulness of a model’s response without penalising conversational elements and captures refusal to address questions outside of the system’s scope of practice. |