When is a medical device including AI subject to the AI Act?
All medical devices that are or which include an AI system, as defined in the AI Act, and which are targeted at the EU market, will be subject to the Act's provisions. "AI System" is broadly defined and might include the use of big data to provide, for example, relatively accurate predictions of an indication, rather than being truly artificial intelligence. As a result, the AI Act brings many medical devices already on the market within its scope.
Medical devices and high- or limited-risk AI systems under the AI Act
Medical devices might be high-risk or otherwise of limited risk, with the former being subject to a complex regulatory regime requiring notified body certification of the AI system, in addition to the medical device regulation requirements.
There are two ways for medical devices to qualify as high-risk AI systems (HRAIS). Firstly, if the device is subject to a notified body conformity assessment under the medical device regulations and that AI system is either:
- intended to be used as a safety component, or
- "is itself a product", such as software as a medical device (SaMD).
Second, some medical devices could be defined as high-risk under Annex III of the AI Act. This might be the case for devices which track emotions, for example, mental health applications using a camera to read facial expressions.
If both definitions apply, then potentially both regimes: (i) high-risk AI system, and (ii) Annex III, will apply, although there are some exemptions from Annex III where the risks are declared by the provider to be low. While there is some dissent on this, there is nothing in the AI Act to indicate that only one regime will apply if both definitions apply.
Manufacturers of medical devices which already include an AI system, and which are classified as high-risk and are placed on the market or put into service before 2 August 2026, do not need to bring those systems into compliance with the AI Act unless they are subject to significant changes in their designs. However, the term "placing on the market" refers to individual devices, not a class of devices and therefore any further devices of that type must comply with the AI Act before they are placed on the market or put into service.
Medical device regulation vs AI Act
At its most basic, software that has clinical application directly on individuals, whether for their diagnosis or treatment, will likely fall within the EU medical device regulations. In the AI Act, medical devices that constitute HRAIS require notified body certification. While all medical devices require a declaration of conformity, only medical devices categorised as HRAIS will require that the provider sign an EU declaration of conformity that is also compliant with the AI Act. For medical device manufacturers of medical devices which qualify as HRAIS, only a single declaration of conformity (DoC) for both sets of legislation will be necessary.
It is well understood that the EU GDPR applies to personal data which can underpin software used in medical devices, particularly SaMD, but the AI Act requires that the provider declares in the DoC that their medical devices qualifying as HRAIS are compliant with EU GDPR. The declaration of conformity for the medical devices that qualify as HRAIS therefore has potential to be invalidated in the event of non-compliance with the GDPR.
Requirements for limited-risk medical devices
While manufacturers of medical devices deemed limited-risk under the AI Act will escape the complexity that comes with regulation by a notified body, there are still provisions that they must comply with in addition to the medical device regulations. All providers of AI systems must ensure "a sufficient level of AI literacy" of their staff and other persons dealing with the operation and use of AI systems". This means manufacturers of medical devices including an AI system will, from 2 February 2025, have to implement additional staff training specifically with respect to that AI system.
They will also be encouraged to voluntarily comply with codes of conduct "encouraged and facilitated" by the AI Office and Member States. It is not clear who will be drafting these, but one would hope that they will be industry specific rather than general. They are due to be finalised by 2 May 2025.
Where any medical device that includes an AI system is intended to interact directly with individuals, then the provider (device manufacturer) must inform the individuals that they are interacting with an AI system unless this is obvious. This could be included in notices that appear as a device or app starts-up, as they must be supplied at the "first interaction or exposure".
The rest of this article details provisions applicable to medical devices that are or include HRAIS.
Obligations for medical devices falling within Annex III
Medical devices that are HRAIS under Annex III are subject to the following obligations:
- If they make decisions related to individuals, those individuals must be informed of the use of the HRAIS.
- Serious incidents that infringe EU obligations intended to protect fundamental rights must be reported to the market surveillance authority (in addition to reporting incidents as required by medical device regulations).
- Should a provider/manufacturer consider that a device covered by Annex III is not high-risk, they must document that assessment and register the system in the EU database.
- Any request from a market surveillance authority to provide documents created or maintained under the AI Act with respect to HRAIS must be responded to.
Provider vs manufacturer
The AI Act uses the term "provider" where the medical device regulations use the term "manufacturer". Their roles are equivalent in that they are the principal actor responsible for the compliance of the design and manufacture or build of the product with the applicable regulations. The AI Act does not have an equivalent of the Annex I General Safety and Performance Requirements (GSPRs) of Regulation (EU) 2017/745 for general medical devices (MDR) or Regulation (EU) 2017/746 on in vitro diagnostic medical devices (IVDR). This is indicative of a regulation for which the detail has not been thought through and/or is targeted so broadly that only guidance specific to the industry sector will be workable. Guidance equivalent to GSPRs is necessary for companies to know how to draft the part of their technical documentation that addresses the HRAIS aspects of their medical devices. This issue is particularly acute for providers/manufacturers with devices already on the market or which will be ready for placing on the market or putting into service on or just after 2 August 2026.
Deployer vs user
Under the medical device regulations, the user of the device (who can be a patient, a healthcare provider or a carer) is the person the regulations are intended to benefit and who therefore has no obligations under those regulations. Under the AI Act there is a different economic operator: the "deployer" who can be an individual or an entity that "uses the AI system under its authority", but not if the device is used in a personal capacity. This definition excludes patients but includes health authorities and hospitals which provide access to or use on patients, devices which include an AI system. The deployer is an economic operator in the terminology of the EU's New Approach legislation and is subject to specific obligations enforced by national competent authorities. Realistically clinics and hospitals will not have the capability or the resources to meet all the AI Act requirements placed upon them where they are using or supplying medical devices which qualify as HRAIS. To ensure a competitive edge, providers/manufacturers should review the deployer obligations in article 26 of the AI Act and provide support for their customers to enable them to meet those requirements.
Quality management system (QMS) for MDR/IVDR vs for AI Act
Manufacturers of medical devices are used to operating and maintaining a quality management system (QMS). The QMS for medical devices extends across the whole organisation and the lifecycle of the device, from design, to processes and procedures for manufacture, quality control, to distribution and post-market surveillance and vigilance. The AI Act specifically permits providers/manufacturers to combine the QMS for AI Act compliance with the QMS for their medical device.
While a QMS for medical devices is about maintaining the quality of those devices, the QMS defined under the AI Act is rather broader, and indeed, is arguably rather a system for compliance with the entirety of the AI Act given that it requires "a strategy for regulatory compliance". A provider/manufacturer of medical devices classed as HRAIS will need to consider the processes and procedures for design, development, verification and validation as they apply to the AI system. This will likely include a lot more detail around data and how it is handled than is usually the case for medical devices, even for SaMD.
Provisions applicable to medical devices which are HRAIS will be in force in under three years. By 2 August 2027, providers must have a certificate of conformity from a notified body to continue to place on the EU market devices which are categorised as HRAIS. The current guidance lacuna is currently best filled by turning to harmonised standards. Some standards specific to AI systems already exist, and many more are under development. Medical device manufacturers might extrapolate the principles of those more familiar medical device standards to the HRAIS. These include ISO 13485 on quality management systems and ISO 14971 on risk management. The equivalent standards for AI systems are ISO 42001:2003 on AI management system, and ISO 23894:2023 – AI – Guidance on risk management respectively.
Clinical investigations/performance studies
The EU medical device regulations provide an exception to the rule that only CE marked devices might be placed on the market for devices used in an authorised clinical investigation or an authorised or notified performance study. Clinical evidence from these studies is for most devices an absolute necessity to confirm that the device is safe and effective as set out in the technical documentation. The AI Act provides for no equivalent exception for medical device HRAIS that qualify as high-risk under Article 6(1). The only option for these devices to be tested after 2 August 2027 but before launch is via an AI regulatory sandbox, the first of which is to be established by competent authorities by 2 August 2026.
For AI systems qualifying as high-risk under Annex III, there is provision for testing under a real-world testing plan that has been authorised by or notified to the market surveillance authority in the Member State. However, this option is only available if the "predictions, recommendations or decisions of the AI system can be effectively reversed and disregarded" – making this not in reality "real world" testing. It would be sensible if the Commission were to either extend this real-world testing to all medical devices categorised as HRAIS and to allow for irreversible application, or to allow testing authorised pursuant to the medical device regulations to be followed as an alternative. For now, manufacturers/providers whose devices qualify as HRAIS under Article 6(1) and not Annex III, or qualify as HRAIS under Annex III but have outputs or effects that are not reversable, would be advised to complete all planned clinical investigations and performance studies by 2 August 2027, when the relevant provisions become enforceable.
Getting ready for compliance
Following on from the extensive regulatory burdens of the MDR and the IVDR, the AI Act brings an unwelcome level of additional regulatory complexity for medical device manufacturers of devices categorised as HRAIS. Given the scrutiny already available under the MDR and IVDR, including for AI elements of any device, this additional regulatory burden is arguably unnecessary.
Medical device manufacturers that include AI in their devices should undertake a mapping exercise to determine if and how the AI Act applies. They will want to take advantage of any overlapping provisions to minimise additional work. For devices categorised as HRAIS there will be additional costs incurred in building AI content into technical documentation and additional notified body fees. Device manufacturers will need to build these costs and the inherent additional delays into their plan for the timing of placing their HRAIS medical devices on the market in the EU after 2 August 2027.
