Artificial intelligence promises to revolutionize healthcare delivery in resource-constrained settings, but a comprehensive systematic review published in Discover Artificial Intelligence exposes a troubling reality: the vast majority of machine learning prediction tools designed for public health lack the rigorous validation and transparency needed for safe clinical deployment—precisely where oversight is weakest.

The analysis, led by Joe Phiri and colleagues, examined 64 studies covering AI prediction models that use electronic health records for public health applications. The findings reveal a stark imbalance: only 18.8% of research was conducted exclusively in low- and middle-income countries (LMICs), despite these regions bearing the greatest burden of preventable disease. High-income countries dominated the literature at 68.8%, with the remaining 12.5% representing mixed-setting studies.

The External Validation Gap

Perhaps most concerning is the near-absence of external validation—testing whether models perform reliably on independent datasets from different institutions or populations. Only 5 of 64 studies (7.8%) included this critical safety check, with just one conducted exclusively in an LMIC setting. Without external validation, health systems have no evidence that an algorithm trained in one hospital will function accurately when deployed elsewhere.

Calibration reporting—which assesses whether predicted probabilities match actual outcomes—appeared in only 4 of 64 studies (6.2%). This gap means clinicians cannot know if a model that predicts a 30% mortality risk truly corresponds to 30 out of 100 patients dying, or whether the algorithm systematically over- or underestimates danger.

Explainability assessments, which help clinicians understand why an algorithm makes specific predictions, were conducted in just 1 of 12 LMIC-exclusive studies (8.3%), compared to 36.4% in high-income research. The review noted that SHAP-based feature importance rankings and post-hoc attribution techniques were most common among the minority of studies that addressed explainability.

Technology Choices Reflect Infrastructure Realities

The algorithmic landscape differs sharply between settings. Deep learning architectures—including recurrent neural networks, LSTM models, and convolutional neural networks—dominated high-income research at 39.1% overall. In contrast, LMIC studies relied primarily on traditional machine learning and ensemble methods, each comprising 33.3% of LMIC-exclusive work.

This divergence reflects practical constraints rather than technological preference. LMIC electronic health systems like OpenMRS, DHIS2, and Zambia's SmartCare were designed for program reporting rather than research-grade data collection. Data completeness frequently fell below 80% for key clinical variables, while high-income platforms like Epic and Cerner provided more standardized multi-institutional datasets.

Traditional methods including logistic regression and generalized linear models appeared in 28.1% of all studies, while ensemble methods such as random forests and gradient boosting comprised 20.3%. Transformer models like Med-BERT, representing 6.2% of studies, appeared exclusively in high-income settings.

Disease Focus and Missed Opportunities

LMIC research concentrated heavily on infectious diseases—HIV and tuberculosis accounted for 45.3% of disease focus—while non-communicable diseases represented only 28.1%. Maternal and child health comprised 14.1%, with health system utilization at 12.5%.

This distribution misaligns with the rapidly growing burden of cardiovascular disease, diabetes, and hypertension in low- and middle-income countries, suggesting both a research gap and a potential opportunity for developers willing to address these conditions.

Implications for Safe Deployment

The structural imbalance identified by Phiri's team—where algorithmic development has dramatically outpaced validation, explainability, and governance—creates deployment risks in settings with limited institutional oversight. The researchers noted that emerging frameworks like TRIPOD+AI and MI-CLAIM offer reporting standards, but adoption remains limited.

For health systems considering AI implementation, the findings suggest prioritizing models with demonstrated calibration and explainability over algorithmic sophistication. For developers, the review validates computationally lighter approaches suited to data sparsity and systematic missingness, while highlighting the urgent need for investment in evaluation rigor rather than architectural complexity.

The review screened 1,248 records from five major databases following PRISMA guidelines, covering publications from January 2018 to March 2025. Its central message is clear: without closing the validation gap, the promise of AI-driven health equity risks becoming a deployment of unproven tools where the consequences of failure fall hardest on those least able to bear them.

Sources

  1. A systematic review of artificial intelligence and machine learning for public health predictions using electronic health records in low and middle income countries with comparative benchmarks from high income settings — Discover Artificial Intelligence
  2. AI health prediction tools lag in low-income countries, review finds — bioengineer.org