Healthcare Giants Push FDA to Make AI Developers Bear the Liability Burden for Medical Algorithms

A coalition of America's most influential healthcare organizations is drawing a clear line: when generative AI medical devices fail, developers—not doctors—should be held accountable. The American Hospital Association, American Medical Association, American Telemedicine Association, and Workgroup for Electronic Data Interchange have jointly urged the FDA to place responsibility for AI safety and performance squarely on the shoulders of manufacturers as the agency prepares new regulatory guidance.

The coordinated push comes as the FDA develops a framework for overseeing generative AI-enabled medical devices, with draft guidance expected "in the coming months," according to Rick Abramson, Director of the FDA's Digital Health Center of Excellence. The stakes are high: these rapidly evolving tools are entering clinical practice at an accelerating pace, yet fundamental questions about liability and oversight remain unresolved.

The Burden Small Practices Cannot Bear

The healthcare groups argue that requiring providers to monitor generative AI medical devices would create an unsustainable burden, particularly for small and rural practices already stretched thin. "Manufacturers should remain responsible for the safety and performance of the devices they sell," the American Telemedicine Association stated in its formal comments to the FDA.

The proposed FDA framework would categorize AI devices into three risk tiers based on their clinical actions: informational tools carrying low risk, systems requiring human action, and autonomous decision-making devices representing the highest risk category. Yet even with this stratification, provider organizations contend that the technical complexity of these systems makes meaningful oversight nearly impossible at the point of care.

The Black Box Problem

A particularly thorny issue has emerged: hospitals report they lack fundamental understanding of how AI models actually function, making it difficult to identify flaws even when robust governance structures exist. This opacity creates a paradox where providers are expected to validate tools whose inner workings remain inscrutable.

The American Medical Association raised additional concerns about the limitations of human oversight, noting that "human-in-the-loop" safeguards may not adequately mitigate risk due to clinician errors or over-reliance on AI recommendations. This observation challenges a common assumption that physician review serves as an effective safety net for AI-generated clinical guidance.

Toward Transparency and Accountability

The Workgroup for Electronic Data Interchange has proposed concrete solutions, suggesting the FDA establish an adverse event reporting process specifically for AI medical devices and develop "nutrition labels" that would clearly communicate how these tools function and what risks they carry. Such standardized disclosures could help bridge the transparency gap that currently hampers both provider oversight and patient understanding.

The regulatory landscape remains in flux following the Trump administration's 2024 rescission of a Biden-era rule that would have required information cards for AI systems. As the FDA works toward its forthcoming guidance, the healthcare industry's unified stance on developer accountability may significantly influence how generative AI medical devices are regulated and who bears legal responsibility when they underperform or cause harm.

Sources

1. Developers, not doctors, should be held accountable for generative AI medical devices, providers say — Healthcare Dive