Peer-Reviewed Publication
medRxiv2025December 30, 2025Journal Article
Rx-LLM: a benchmarking suite to evaluate safe large language model performance for medication-related tasks.
Xingmeng Zhao1, Kaitlin Blotske1, Moriah Cargile2, Adeleine Tilley2, Brian Murray2, Yanjun Gao1, Kelli Henry1, Susan E Smith3, Erin F Barreto4, Seth Bauer5, Sunghwan Sohn4, Tianming Liu6, Tell Bennett7, Mitch Cohen8, Andrea Sikora1,9
1University of Colorado School of Medicine, Department of Biomedical Informatics.
2University of Colorado Anschutz Medical Campus, Skaggs School of Pharmacy and Pharmaceutical Sciences. Department of Clinical Pharmacy, Aurora, CO, USA.
3University of Georgia College of Pharmacy, Department of Clinical and Administrative Pharmacy, Athens, GA, USA.
4Mayo Clinic, Rochester, MN, USA.
5Cleveland Clinic, Cleveland, OH, USA.
6Department of Computer Science, University of Georgia, Athens, GA.
7University of Colorado School of Medicine, Department of Biomedical Informatics Aurora, CO, USA.
8University of Colorado Anschutz Medical Campus, Department of Surgery.
9University of Georgia College of Pharmacy, Department of Clinical and Administrative Pharmacy, Augusta, GA, USA.
Abstract
BACKGROUND: For large language models (LLMs) to reach their potential as information technology tools that make medication use safer, clinically relevant benchmarks capable of automated grading and designed specifically to measure the performance of LLMs for medication tasks are required. The purpose of this study was to design a suite of benchmarking tests reflective of Comprehensive Medication M…
Create a free account to keep reading
Free members get 10 full research views every month across publications, clinical trials, FDA clearances, adverse events, and NIH grants. No credit card required.
Want unlimited research access? See Pro plans