KFF Report Warns AI Could Exacerbate Racial Health Disparities Without Targeted Intervention

A comprehensive report from KFF (Kaiser Family Foundation) has found that AI systems deployed in clinical settings risk exacerbating existing racial and ethnic health disparities unless developers, health systems, and regulators take deliberate steps to address algorithmic bias at every stage of the AI lifecycle — from data collection through post-deployment monitoring.^[1]

The report, which reviewed more than 200 peer-reviewed studies and regulatory submissions, identified three primary mechanisms through which AI can amplify health disparities: biased training data that underrepresents minority populations, validation studies conducted predominantly on white patient populations, and deployment in health systems that already serve predominantly white or higher-income patients.^[1]

The Training Data Problem

Many clinical AI models are trained on data from large academic medical centers, which tend to serve patient populations that are wealthier, more educated, and less racially diverse than the general U.S. population. When these models are deployed in safety-net hospitals or community health centers that serve predominantly Black, Hispanic, or low-income patients, their performance often degrades — sometimes dramatically. A 2023 study published in Nature Medicine found that a widely used sepsis prediction algorithm performed significantly worse in Black patients than in white patients, with a false negative rate that was 3.5 times higher in the Black patient population.^[2]

Regulatory Response

The FDA has begun requiring developers of high-risk AI medical devices to include subgroup performance analyses stratified by race, ethnicity, sex, and age in their 510(k) and De Novo submissions. The agency's 2024 AI/ML action plan included a commitment to develop guidance on equity-focused validation requirements, and the HALO platform launched in May 2026 includes tools for monitoring subgroup performance in post-market surveillance.^[3]

Recommendations

The KFF report makes several recommendations for health systems deploying AI: require vendors to provide subgroup performance data before procurement, conduct local validation studies on the health system's own patient population before clinical deployment, establish ongoing monitoring protocols that track performance by demographic group, and include community representatives in AI governance committees. The report also recommends that CMS tie reimbursement for AI-assisted services to demonstrated equity in performance across patient populations.^[1]

References

1. KFF. "The Growing Use of Artificial Intelligence in Health Care and Implications for Disparities." May 2026. https://www.kff.org/racial-equity-and-health-policy/the-growing-use-of-artificial-intelligence-in-health-care-and-implications-for-disparities/
2. Obermeyer Z, et al. "Dissecting racial bias in an algorithm used to manage the health of populations." Science. 2019. https://www.science.org/doi/10.1126/science.aax2342
3. FDA. "Artificial Intelligence and Machine Learning Action Plan." 2024. https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles