Health AI Central
Peer-Reviewed Publication
J Am Coll Radiol2026September 3, 2026Journal Article

AI Monitoring AI with LLMs: The American College of Radiology's Imaging AI Registry.

Kendall Schmidt1, Laura Brink1, Neil Bhatia2, Jason Yao3, Ahmed-Zayn Mohamed4, Alex Chan5, Hana L Haver6, Laura Coombs7, Woojin Kim8, Tessa Cook9, Keith Dreyer10, Christoph Wald11, Bernardo Bizzo12
1Senior Data Scientist, American College of Radiology, Reston, VA, USA.
2Chief Radiology Resident Department of Radiology, Brigham & Women's Hospital, Harvard Medical School, Boston, Massachusetts, USA, Data Science Institute Scientist, American College of Radiology, Reston, VA, USA.
3Radiology Resident, Department of Medical Imaging, McMaster University, Hamilton, Ontario, CA, Data Science Institute Scientist, American College of Radiology, Reston, VA, USA.
4Radiology Resident, Department of Radiology, University of South Florida, Morsani College of Medicine, Tampa, Florida, USA, Data Science Institute Scientist, American College of Radiology, Reston, VA, USA.
5Radiologist, Department of Radiology, Mayo Clinic, Rochester, Minnesota, USA, Data Science Institute Scientist, American College of Radiology, Reston, VA, USA.
6Associate Director of Artificial Intelligence, Mosaic Clinical Technologies, Nashville, TN, USA, Data Science Institute Scientist, American College of Radiology, Reston, VA, USA.
7Data Science and Informatics, American College of Radiology, Reston, VA.
8Chief Medical Officer, Data Science Institute, Reston, VA, USA, Chief Strategy officer & Chief Medical Information Officer, HOPPR, Chicago, IL, USA, MSK Radiologist, VA Medical Center, Palo Alto, CA, USA.
9Vice Chair ACR Commission on Informatics, Reston, VA, USA, Associate Professor and Vice Chair of Practice Transformation, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA, USA.
10Chief Science Officer, Data Science Institute, Reston, VA, USA, Chief Data Science Officer, Vice Chairman of Radiology, Mass General Brigham, Boston, MA, USA.
11Vice Chair, ACR Board of Chancellors; Chair ACR Commission on Informatics, Reston, VA, USA, Professor of Radiology & SA Consultant Radiologist, Department of Radiology, Mayo Clinic, Rochester, MN, USA.
12Associate Chief Science Officer, Data Science Institute, Reston, VA, USA, Senior Director, Mass General Brigham, Boston, MA, USA.
PubMed DOI

Abstract

OBJECTIVE: To describe the technical workflow enabling scalable automated artificial intelligence (AI) monitoring in the first national imaging AI registry, Assess-AI. MATERIALS AND METHODS: Large language model (LLM) prompts are developed to extract clinically relevant findings from radiology reports through collaboration between data scientists and subspecialty radiologists. Prompts are optimiz…

Create a free account to keep reading

Free members get 10 full research views every month across publications, clinical trials, FDA clearances, adverse events, and NIH grants. No credit card required.

Create Free AccountSign In

Want unlimited research access? See Pro plans

Data Accuracy Notice: Research intelligence on Health AI Central is aggregated from public sources (PubMed, ClinicalTrials.gov, FDA, NIH, CMS, and others) and refreshed nightly. Classifications and derived metrics are produced by automated methods described in our Methodology. We recommend verifying critical data points against the primary sources before making decisions.