AI-based medical devices are becoming more prevalent, and ensuring their safety, effectiveness, and usability is critical. The Good Machine Learning Practice for Medical Device Development: Guiding Principles (2024 Draft), aims to establish a framework to ensure these devices meet high standards throughout their lifecycle.
In this article, we explore key aspects of the document and analyze what it means for usability, regulatory compliance, and human-AI interaction in medical devices. By the end, you’ll have a clear understanding of usability implications for medical devices based on this guideline.
This post is part of our AI & Usability in Medical Devices Series, where we examine major regulatory guidelines shaping AI usability in healthcare. Each part of the series provides deep insights into different frameworks, building towards a comprehensive overview of global AI regulations for medical devices.
Let’s start.
Good Machine Learning Practice for Medical Device Development: Guiding Principles
The Good Machine Learning Practice (GMLP) for Medical Device Development: Guiding Principles (2024 Draft) was issued by the International Medical Device Regulators Forum (IMDRF) in June 2024. It provides 10 guiding principles aimed at ensuring AI-based medical devices are safe, effective, and high-quality throughout their lifecycle.
The document is part of ongoing global efforts to harmonize AI regulations in medical devices and builds upon previous AI/ML guidance from the FDA, Health Canada, and the UK’s MHRA. While it is not a legally binding regulation, it serves as a framework for best practices that regulators may incorporate into future AI policies.
🔗 Read the full document: IMDRF GMLP 2024 Draft
🔗 Read the previous AI/ML guidance: FDA & MHRA GMLP 2021
Implications on Usability for Medical Devices
Below, we highlight the most relevant sections of the IMDRF GMLP 2024 Draft and analyze their implications for usability, regulatory compliance, and human-AI interaction.
Principle 1 – The device’s intended use/intended purpose is well understood, and multidisciplinary expertise is leveraged throughout the total product life cycle:
📖 Original citation: „In-depth understanding of a medical device’s intended use/ intended purpose including context of use within the clinical workflow, and the desired benefits and associated patient risks, can help ensure that AI-enabled medical devices address clinically meaningful needs over the total product life cycle of the device. […]“
Implications on Usability:
- Usability must be integrated from the start – regulators will expect AI-based devices to fit seamlessly into real-world clinical workflows, meaning usability cannot be an afterthought.
- Regulatory justifications must demonstrate how users interact with AI over time, ensuring that usability validation includes both initial training and long-term adoption.
- Multidisciplinary collaboration will become a regulatory expectation – human factors, AI engineers, and clinical teams must work together to ensure intended use translates into real-world usability.
Principle 2 – Good software engineering, medical device design, and security practices are implemented:
📖 Original citation: “Model design is implemented and maintained with attention to the fundamentals: robust software engineering practices, usability, data quality assurance, data management, cybersecurity, and quality management practices. These practices include methodical risk management and design processes that can appropriately record and communicate decisions and rationale […]”
Implications on Usability:
- Usability is explicitly named as a fundamental requirement – regulators expect usability to be documented and integrated into AI software engineering.
- Cybersecurity-related usability will gain more scrutiny – error prevention mechanisms such as intuitive authentication, alert design, and user fallback strategies will be critical.
- Post-market AI usability monitoring will become part of compliance – manufacturers must track how users interact with AI as software updates or retraining occur, ensuring usability remains intact.
Principle 7 – Performance is assessed with a focus on the human-AI team in the intended use environment:
📖 Original citation: “The performance of the model outputs is assessed in the context of the intended use environment and clinical workflow. Human factors considerations are addressed, including user skills, user expertise, user understanding of the model outputs and limitations, and user error, for normal use and reasonably foreseeable misuse.”
Implications on Usability:
- Usability validation must shift from UI design to human-AI collaboration – regulators will expect manufacturers to assess how well users understand and interact with AI outputs in real-world workflows.
- New usability metrics will be required – beyond traditional effectiveness and efficiency measures, AI usability must now consider trust in AI decisions, interpretability of model outputs, and human-AI decision-making efficiency.
- Error mitigation strategies must be usability-tested – regulatory focus will extend beyond AI accuracy to how users recover from errors and how AI designs minimize automation complacency and over-reliance.
- “User error” vs. “use error” – an inconsistency? The term “user error” appears in the draft, while established human factors terminology typically refers to “use error” (an error occurring because of system design, not user mistakes). If this phrasing is intentional, it could imply a shifting regulatory focus toward user responsibility, rather than solely system design. However, it’s more likely an oversight, and clarification may be needed in the final version.
Principle 9 – Users are provided clear, essential information:
📖 Original citation: “The intended audience (such as health care professionals or patients) are provided clear, contextually relevant information appropriate to their needs. This includes the product’s intended use/intended purpose and indications for use, benefits and risks, performance of the model for appropriate subgroups, the study methodology, characteristics of the data used to train and test the model, acceptable inputs, known limitations, user interface interpretation, clinical workflow integration of the model, and to the extent possible the basis for model output. Users are also made aware of the scope and timing of device modifications and updates. They are provided a means to communicate product concerns to the manufacturer.”
Implications on Usability:
- Regulatory focus on AI self-descriptiveness will require UX transparency – manufacturers must ensure AI-generated outputs are intuitive, clinically relevant, and structured for clear decision-making.
- Poorly designed AI outputs could be classified as usability failures, leading to regulatory rejection or increased compliance hurdles.
- AI updates must be communicated effectively to users – regulators are signaling that usability extends beyond initial training. Users must be able to understand how AI models evolve and anticipate any workflow changes.
Discussion: Usability as a Core Consideration in AI Regulation
The IMDRF 2024 GMLP draft signals a growing regulatory focus on usability in AI-based medical devices. While still a draft, it suggests that usability will become a compliance factor across the entire product lifecycle – from design and validation to post-market monitoring.
A key shift is the emphasis on human-AI collaboration. Usability evaluations will likely need to go beyond interface design and assess how users interpret AI outputs, build trust in AI-driven decisions, and recover from errors. Regulators are also highlighting AI self-descriptiveness as a usability requirement, meaning unclear or overly complex AI outputs could become a regulatory risk.
As this is still a draft, the final regulatory expectations may evolve. However, companies developing AI-based medical devices should start aligning with these principles now to stay ahead of future compliance challenges.
Where to go from here?
- 📩 You can sign up for our newsletter here to receive notifications when we release a new part.
- 🤝 If you need help integrating usability/user interface design into your (AI-based) medical device, feel free to contact us.
