Manufacturers of Machine Learning-Enabled Medical Devices (MLMDs) face a key challenge: existing requirements and processes on the notification and management of ‘substantial’ and ‘significant’ changes to medical devices need adapting for MLMDs. The MHRA, together with the U.S. Food and Drug Administration (FDA) and Health Canada, has published five guiding principles on predetermined change control plans (PCCPs) to help resolve this tension.
What is the tension between AI and change management for medical devices?
Some AI models are static – once they are on the market, they do not update. However, some models do retrain, in batches or continuously. Those models that retrain in batches or continuously may pose safety concerns if small changes in input data or the model may cause dramatically different outputs.
Under medical device legislation in the UK and EU, manufacturers of MLMDs will need to notify the relevant notified body of ‘substantial’ and ‘significant’ changes to a device and there are detailed change control requirements. This inflexible approach is at odds with the more dynamic software and AI world, where models may be constantly tweaked over their lifecycle. How can manufacturers of a continuously training model have certainty that a change is not ‘substantial’ or ‘significant’, and therefore not triggering enhanced premarket review requirements?
Are predetermined change control plans (PCCPs) the answer?
PCCPs can help to manage certain device changes where regulatory authorization before marketing is typically required. A PCCP is a plan, proposed by a manufacturer, that specifies certain planned modifications to a device; the protocol for implementing and controlling those modifications; and the impacts from modifications. The MHRA has published five principles for PCCPs:
- A PCCP must be focused and bounded: A PCCP describes specific changes that a manufacturer intends to implement. These changes are limited to modifications within the intended use or intended purpose of the original MLMD. This can include the extent of planned changes, and plans to safely modify the device within the bounds of the PCCP, including methods for verifying and validating the changes and mechanisms to detect and revert a change that fails to meet specified performance criteria.
- The intent, design and implementation of a PCCP must reflect a risk-based approach from inception, through to implementation and use.
- PCCPs must be evidence-based: This is to ensure the ongoing safety and effectiveness of the device, demonstrate that the benefits outweigh the associated risks, and establish that the risks are adequately managed and controlled.
- Best practice is to provide information to ensure ongoing transparency to users and other stakeholders: This helps ensure that stakeholders stay aware of the device’s performance and use before and after changes are implemented.
- PCCPs should be created and used from a Total Product Lifecycle (TPLC) perspective: This uses and supports existing regulatory, quality, and risk management measures throughout the TPLC to ensure device safety by monitoring, reporting and responding to safety concerns.
These guiding principles are intended to lay a foundation for PCCPs and encourage international harmonization.
