MedPromptEval: A Comprehensive Framework for Systematic Evaluation of Clinical Question Answering Systems.

Al Rahrooh¹, Anders O Garlid¹, Panayiotis Petousis¹, Arthur Funnell¹, Alex A T Bui¹

¹Medical & Imaging Informatics Group, Department of Radiological Sciences, David Geffen School of Medicine, University of California Los Angeles, Los Angeles, CA.

PubMed DOI

Abstract

Clinical deployment of large language models (LLMs) faces critical challenges, including inconsistent prompt performance, variable model behavior, and a lack of standardized evaluation methodologies. We present MedPromptEval, a framework that systematically evaluates LLM-prompt combinations across clinically relevant dimensions. This framework automatically generates diverse prompt types, orchestr…

Create a free account to keep reading

Free members get 10 full research views every month across publications, clinical trials, FDA clearances, adverse events, and NIH grants. No credit card required.

Create Free Account Sign In

Want unlimited research access? See Pro plans

Data Accuracy Notice: Research intelligence on Health AI Central is aggregated from public sources (PubMed, ClinicalTrials.gov, FDA, NIH, CMS, and others) and refreshed nightly. Classifications and derived metrics are produced by automated methods described in our Methodology. We recommend verifying critical data points against the primary sources before making decisions.