Written and maintained by CASRAI Editorial Board
Last updated
Short answer: Good Machine Learning Practice (GMLP) is the international regulatory framework for developing, validating, and monitoring artificial intelligence and machine learning (AI/ML)-enabled medical devices across the Total Product Life Cycle (TPLC). Jointly established in October 2021 by the U.S. Food and Drug Administration (FDA), Health Canada, and the UK Medicines and Healthcare products Regulatory Agency (MHRA)—and subsequently harmonized globally through the International Medical Device Regulators Forum (IMDRF)—GMLP defines 10 guiding principles spanning multidisciplinary lifecycle integration, software engineering and cybersecurity, demographic representativeness, strict independence between training and test datasets, reference standard rigor, human-AI team evaluation, user transparency, and postmarket retraining governance.
What are the 10 guiding principles of Good Machine Learning Practice (GMLP)?
Just as Good Clinical Practice (ICH E6 GCP) governs clinical trials and Good Manufacturing Practice (21 CFR Part 820 / ISO 13485) governs device production, the 10 GMLP Guiding Principles set the baseline expectations that regulatory reviewers apply when evaluating Software as a Medical Device (SaMD) and AI-enabled hardware submissions:
| Principle # | GMLP Guiding Principle | Required Engineering & Clinical Submission Evidence |
|---|---|---|
| 1 | Multi-Disciplinary Expertise Is Leveraged Throughout the Total Product Life Cycle | Documented involvement of practicing clinicians, human factors engineers, biostatisticians, and ML engineers from hazard identification through postmarket surveillance |
| 2 | Good Software Engineering and Security Practices Are Implemented | IEC 62304 software lifecycle traceability, deterministic data provenance, model weight version control, and FDA premarket cybersecurity threat modeling |
| 3 | Clinical Study Participants and Data Sets Are Representative of the Intended Population | Demographic, clinical, and equipment-vendor stratification across age, sex, race/ethnicity, disease prevalence, and care setting matching the labeled intended use |
| 4 | Training Data Sets Are Independent of Test Sets | Strict patient-level, site-level, and temporal separation between training/tuning data and locked clinical validation test sets to prevent data leakage |
| 5 | Selected Reference Datasets Are Based Upon Best Available Methods | Clearly defined ground-truth adjudication (e.g., histopathology gold standard or multi-specialist blinded consensus panel with inter-reader reliability metrics) |
| 6 | Model Design Is Tailored to the Available Data and Reflects the Intended Use of the Device | Justification of model architecture, loss functions, uncertainty quantification, and operating threshold selection (sensitivity vs. specificity trade-offs) for the clinical task |
| 7 | Focus Is Placed on the Performance of the Human-AI Team | Multi-reader multi-case (MRMC) or human factors validation testing measuring clinician + AI performance together rather than standalone algorithm accuracy alone |
| 8 | Testing Demonstrates Device Performance During Clinically Relevant Conditions | Subgroup performance tables, scanner/sensor stress testing, signal noise/artifact robustness, and out-of-distribution handling in real-world clinical workflows |
| 9 | Users Are Provided Clear, Essential Information | Model card / labeling disclosures detailing training cohort demographics, known failure modes, input requirements, and how to interpret model outputs |
| 10 | Deployed Models Are Monitored for Performance and Re-Training Risks Are Managed | Postmarket AI model drift monitoring protocols and a cleared Predetermined Change Control Plan (PCCP) prevented catastrophic forgetting or bias amplification |
What are the GMLP expectations for training/test data independence and human-AI teaming?
In regulatory deficiency letters for 510(k), De Novo, and PMA submissions, two GMLP principles account for the majority of methodological rejections:
Principle 4: Strict Independence of Training and Test Datasets
Regulators require that the standalone clinical validation dataset (test set) be completely isolated from the algorithm development and hyperparameter tuning datasets:
- Patient-Level Partitioning: No patient whose images, waveforms, or EHR encounters appeared in the training or internal validation/tuning cohorts may appear in the final test set. Splitting at the image or encounter level while allowing the same patient to span both sets causes optimistic data leakage.
- Site-Level (External) Independence: To prove generalizability across hospital workflows and acquisition hardware, FDA CDRH routinely requires that pivotal test data originate from clinical sites and scanner models that were not used to train the algorithm.
- Audit Controls on Test Set Reuse: Once an algorithm is evaluated against a locked validation dataset, iteratively modifying the model based on those test errors compromises statistical independence unless a fresh holdout cohort or pre-specified statistical alpha-spending plan is used.
Principle 7 & 9: Human-AI Teaming and the 2024 Transparency Principles
Most clinical AI devices are cleared as computer-aided detection (CADe), diagnosis (CADx), or triage (CADt) tools that assist—rather than replace—a licensed clinician. Under GMLP Principle 7 and the June 2024 FDA/Health Canada/MHRA Transparency for Machine Learning-Enabled Medical Devices: Guiding Principles:
- Evaluating the Clinician-AI Dyad: Sponsors must evaluate whether the user interface induces automation bias (clinicians uncritically accepting false-positive or false-negative AI outputs) or alert fatigue. For diagnostic aids, FDA frequently requires a fully crossed or split-plot Multi-Reader Multi-Case (MRMC) study comparing clinician diagnostic performance aided by the AI against unaided clinician performance.
- Actionable User Transparency: Device labeling and on-screen displays must communicate the basis of the prediction, confidence intervals or quality warnings when input signals are degraded, and populations where the model has not been validated—aligning with TRIPOD+AI reporting standards.
How do FDA, Health Canada, and MHRA apply GMLP to medical device development?
While the 10 GMLP Guiding Principles are non-binding policy statements on their own, the three agencies and the IMDRF operationalize them through binding regulatory submission pathways and consensus standards:
- Tripartite PCCP Alignment (2023–2025): Building directly on GMLP Principle 10, the FDA, Health Canada, and MHRA released joint guiding principles for Predetermined Change Control Plans (PCCPs), allowing manufacturers to pre-specify data collection, retraining, and verification protocols in their initial marketing submission so bounded postmarket ML updates can be deployed safely.
- Consensus Standards Integration: Reviewers map GMLP compliance onto recognized consensus standards, including ISO 14971 and AAMI TIR34971 (Application of ISO 14971 to Machine Learning in Artificial Intelligence), IEC 62304 (Medical Device Software Lifecycle Processes), and IEC 62366-1 (Usability Engineering).
Frequently asked questions
What are the 10 guiding principles of Good Machine Learning Practice (GMLP)?
The 10 GMLP guiding principles cover: (1) multidisciplinary expertise across the total product life cycle, (2) good software engineering and security practices, (3) representative clinical study participants and datasets, (4) independence of training and test datasets, (5) best-available reference standards, (6) model design tailored to data and intended use, (7) focus on the performance of the human-AI team, (8) testing under clinically relevant conditions, (9) clear, essential information for users, and (10) monitoring deployed models and managing retraining risks.
How do FDA, Health Canada, and MHRA apply GMLP to medical device development?
The FDA, Health Canada, and UK MHRA use the 10 GMLP guiding principles—along with their follow-on 2023 Predetermined Change Control Plan (PCCP) principles and 2024 Transparency principles—as the review framework for 510(k), De Novo, PMA, Health Canada Medical Device Licence, and UKCA/CE marking submissions involving AI/ML-enabled medical devices.
What are the GMLP expectations for training/test data independence and human-AI teaming?
Under Principle 4, validation test datasets must be separated from training and tuning datasets at the patient, clinical site, and acquisition-device levels so that zero data leakage occurs. Under Principle 7, validation must measure the combined performance of the clinician and AI working together (such as through Multi-Reader Multi-Case studies and human factors testing) to guard against automation bias.
Sources
- U.S. FDA, Health Canada, and UK MHRA — Good Machine Learning Practice for Medical Device Development: Guiding Principles (October 2021)
- U.S. FDA, Health Canada, and UK MHRA — Transparency for Machine Learning-Enabled Medical Devices: Guiding Principles (June 2024)
- U.S. FDA, Health Canada, and UK MHRA — Predetermined Change Control Plans for Machine Learning-Enabled Medical Devices: Guiding Principles (October 2023)
