Medical artificial intelligence is rapidly becoming a fixture in Australian clinical practice—from BreastScreen NSW's deployment of Lunit Insight MMG for mammogram interpretation to AI-assisted diagnostic tools in emergency departments. But beneath the promise of enhanced efficiency lies a troubling reality: these systems may be systematically disadvantaging the very populations most in need of equitable care.

According to experts interviewed by InSight+ (MJA), bias embedded in AI training data and algorithms risks perpetuating healthcare inequities, particularly for Aboriginal and Torres Strait Islander peoples, ethnic minorities, and other marginalized groups. The concern is not theoretical. A landmark US study revealed an algorithm using healthcare costs as a proxy for health needs systematically allocated fewer resources to African American patients—not because of their clinical requirements, but because historical spending patterns reflected unequal access rather than true need.

When Machines Learn Human Prejudice

The Australian Journal of General Practice reports that implicit bias already leads to underestimation of Aboriginal and Torres Strait Islander people's pain experience, causing treatment delays. When AI systems are trained on data reflecting these existing disparities, they risk amplifying rather than correcting them.

Associate Professor Sanjay Warrier, a Sydney-based breast oncology and oncoplastic surgeon, notes that AI diagnostic tools perform unevenly when trained on non-representative datasets. Factors like breast density, age, and ethnic background can significantly affect accuracy—yet many commercial systems lack adequate diversity in their training populations.

Associate Professor Sonika Tyagi from RMIT University's School of Computing Technologies identifies three critical bias types: measurement bias (how data is collected), interpretation bias (how results are understood), and data-level bias (what populations are represented). As the Harvard Data Science Review succinctly states: "A biased world produces biased data."

The Clinical Safeguards That Matter

The Australian Commission on Safety and Quality in Health Care released Pragmatic AI Guidance for Clinicians in 2025, emphasizing that training data must reflect patient population diversity. But technical solutions alone are insufficient.

Professor Ian Scott, Clinical Consultant in AI at Metro South Hospital and Health Service Digital Health and Informatics Directorate, recommends context-aware prompting and retrieval-augmented generation (RAG) as mitigation strategies. These approaches help ground AI outputs in verified clinical knowledge rather than potentially biased pattern recognition.

Yet the most critical safeguard remains distinctly human. A 2026 JACEP paper defines automation bias as clinicians over-relying on AI recommendations despite questionable outputs—a phenomenon with potentially serious consequences. AHPRA guidelines explicitly mandate that practitioners apply human judgment to any AI output.

Professor Liz Marles, the Commission's Clinical Director and a GP, advises viewing AI as a prompt or tool rather than an authority. Dr. Warrier's practice is building an AI tool using Claude to mirror multidisciplinary team clinical decision-making, but emphasizes it will augment rather than replace human expertise.

A Socio-Technical Challenge

Scientia Professor Manju Ahuja from UNSW Business School frames AI bias as a socio-technical issue requiring consideration of human bias throughout the AI product lifecycle. This means involving diverse stakeholders—including patient advocates from affected communities—in development processes.

The CSIRO warns that biased algorithms can delay diagnoses, overlook symptoms, and reinforce inequalities. For clinicians, this translates to a professional obligation: actively audit AI tool outputs for bias, particularly when treating underrepresented populations, and maintain clinical judgment as the primary decision-making authority.

As medical AI becomes ubiquitous, the question is not whether bias exists—it demonstrably does—but whether healthcare systems will implement the human oversight necessary to prevent technology from deepening the inequities it was meant to solve.

Sources

  1. What doctors should know about bias in medical AI — InSight+