When 94 thyroid cancer specialists reviewed 100 clinical cases, they unanimously agreed on treatment in only half of them. When six leading AI models reviewed the same cases, their biopsy recommendations ranged from 46% to 74%—a 28-percentage-point spread that highlights a troubling paradox at the heart of medical AI validation.

The study, published in The Journal of Clinical Endocrinology & Metabolism, examined how large language models perform on a task where clinical judgment is genuinely uncertain: deciding whether to biopsy thyroid nodules. The findings reveal not just AI inconsistency, but a deeper challenge—when physicians themselves disagree, how can we determine if AI recommendations are "correct"?

Wide Variation Across Models and Clinicians

Researchers led by James H. Flory, MD, MSCE—formerly of Weill Cornell Medicine and Memorial Sloan Kettering Cancer Center, currently at Regeneron—presented 100 clinical vignettes of thyroid nodules varying in size and risk factors to both clinicians and AI models. The clinicians, all treating at least 10 thyroid patients annually, recommended biopsies in 58% of cases on average.

Among the AI models tested, recommendations varied dramatically: Claude Sonnet 3.5 recommended biopsies in just 46% of cases, while Llama 3.2-90B suggested them in 74%. GPT-5 landed at 70%. The study also evaluated DeepSeek-R1 and three versions of Claude Sonnet.

The clinicians themselves showed only moderate inter-rater consistency, with a Krippendorff's alpha of 0.435. This limited consensus among experts creates what the research exposes as a fundamental validation challenge for clinical AI systems.

When AI Gets It Wrong—and When We Can Tell

Despite high internal consistency—GPT-5 gave unanimous responses when shown the same vignette multiple times in 89% of cases—the models produced concerning errors. Expert reviewers identified 23 discordant LLM responses, with eight containing serious flaws.

Claude Sonnet 3.5 proved particularly problematic in edge cases: it recommended against biopsy in four of 10 cases where all clinicians unanimously recommended the procedure, and suggested biopsy in two of eight cases where all clinicians agreed it was unnecessary.

Agreement with ACR TI-RADS guidelines—established standards for thyroid nodule management—varied by model, with clinicians showing moderate agreement themselves. This suggests the models aren't simply learning physician preferences, but may be applying different interpretive frameworks to ambiguous clinical scenarios.

The Validation Paradox and What It Means for Clinical AI

The research reveals a critical gap in how medical AI is evaluated. Traditional benchmarking against clinician decisions assumes physicians represent a reliable gold standard. But when legitimate clinical uncertainty exists—as it does for intermediate-risk thyroid nodules—that assumption breaks down.

For healthcare providers, the study indicates that LLMs cannot yet serve as authoritative decision-makers in ambiguous clinical scenarios, though they may offer supplementary support for guideline lookup. For AI developers, it underscores the need for validation frameworks that account for legitimate clinical uncertainty and clearly define which specific scenarios their models can reliably address.

For patients with borderline thyroid nodules, the implications are tangible: the AI tool a provider uses could influence whether they receive a biopsy, potentially leading to either unnecessary procedures or missed cancers. The 28-percentage-point spread in recommendations across models suggests that in clinical decision support, model selection matters as much as the decision to use AI at all.

Sources

  1. Large language models provide variable responses on whether to biopsy thyroid nodules — Healio