New rules proposed for healthcare AI devices

AI devices need regulatory approaches tailored to their unique features. The future framework should support proportionate and tailored oversight across the product lifecycle, including clear routes to market, flexible mechanisms to support rapid device iteration and stronger use of real-world evidence to support ongoing assurance, the report reads.

The commission called for clearer rules on when an AI product should be regulated as a medical device and how much oversight it should receive, based on potential risk to patients and its intended purpose. “Intended purpose should explicitly include device design and functionality in addition to a manufacturer’s claims and promotional materials,” the report reads.

It is important to understand how and when the specifics of how a device has been designed to function is factored into understanding its intended use, especially if that design is inconsistent with specific claims or promotional materials. Clear guidance and resources are needed to support more consistent and ongoing evaluation and understanding of a device’s intended purpose.

The commission also recommended allowing some AI products to enter the market in stages so they can be used under specific conditions while developers continue gathering evidence about their safety and effectiveness.

Regulating AI Devices

The report calls for greater use of real-world data to monitor how AI performs after deployment and for regulatory testing environments where developers and regulators can evaluate new technologies before they are widely introduced into the sector.

The commission said regulators should consider whether AI works safely and effectively across different patient populations, and the framework should also account for “international regulatory harmonization.”

“The MHRA should develop and introduce recognition pathways for software and AI-enabled medical devices with selected international regulators where it is in the best interest of patients. These pathways should include review points and break clauses, so they remain responsive to emerging capabilities and risks, while supporting investment and market growth and ensuring patient safety,” the report reads.

The guidance should also address technologies such as agentic AI, which the commission defines as systems that can autonomously pursue goals and coordinate tasks with limited direct human oversight.

Additionally, healthcare professionals should receive training to understand how to use AI technologies safely, the commission said, including the AI’s limitations and potential risks.

The commission recommended initial professional education, postgraduate training and continuing professional development around AI technology.

Healthcare providers would also be responsible for ensuring staff receive training specific to the AI technologies they use.

“Healthcare providers must ensure that professionals and their wider staff are well equipped to use these technologies by undertaking appropriate training, including technology-specific training aligned with the AI technologies (including those with automated decision-making functionality) used in their practice,” the report states.

Shared Responsibility Needed

The commission said that responsibility for AI safety should be shared among manufacturers, healthcare providers, clinicians, regulators and policymakers.

It also calls for agreements outlining who is responsible for safeguards, including cybersecurity and staff training, and greater clarity around liability when AI use contributes to patient harm.

The commission recommended that patients have access to information about how AI influenced their care and ways to seek redress when something goes wrong.

“Liability for AI in healthcare is complex and evolving. Wider legal reform may be needed, is likely to take time, and will require cooperation beyond the National Commission,” the report reads. “Any future legal reform should create the conditions that encourage the responsible innovation of AI-enabled healthcare technologies, while providing appropriate and fair routes to redress so affected parties can challenge harmful AI-enabled outcomes. In the meantime, the health system should take practical steps to ensure that the adoption of AI technologies does not weaken patients’ access to redress.”

Patient Involvement Matters

It also says patients and the public should have an ongoing role in AI regulation decisions, and that patients and healthcare professionals should be periodically surveyed about their attitudes toward healthcare AI.

The report recommends stronger cybersecurity requirements and clearer guidance for consumer health apps and wearables.

The 44 recommendations are advisory and do not create new regulatory requirements.

The commission said, “A cross-government response will follow separately, setting out how government and system partners will consider and take forward the recommendations.”