Written and maintained by CASRAI Editorial Board
Last updated
CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence) is an international consensus reporting guideline that extends the core CONSORT 2010 statement to randomized controlled trials (RCTs) evaluating clinical interventions with an artificial intelligence (AI) or machine learning (ML) component. Published simultaneously in September 2020 in Nature Medicine, The BMJ, and The Lancet Digital Health by the CONSORT-AI and SPIRIT-AI Steering Group (led by Xiaoxuan Liu, Samantha Cruz Rivera, David Moher, Melanie J. Calvert, and Alastair K. Denniston), the CONSORT-AI extension adds 14 new checklist items (11 extensions and 3 elaborations) to the standard 25-item CONSORT 2010 checklist. These items address reproducibility and safety risks unique to clinical AI systems—specifically algorithm version drift, input data acquisition and quality filtering, human-in-the-loop workflow integration, and case-level analysis of algorithmic performance errors.
What Is the CONSORT-AI Extension and What Types of Clinical AI Trials Must Follow It?
While early machine learning studies in medicine focused largely on retrospective diagnostic accuracy or offline prognostic modeling, regulatory clearance and clinical adoption increasingly require prospective randomized controlled trials demonstrating that deploying an AI system in live clinical care improves patient outcomes or operational safety compared with standard care. Standard CONSORT 2010 items do not capture critical technical variables that determine whether an AI trial’s results generalize to another hospital—such as what happens when a wearable sensor or fundus camera submits a low-quality image, what training clinicians received to interpret probability scores, or whether the neural network weights were updated mid-trial.
CONSORT-AI applies to any randomized trial (including parallel-group, cluster-randomized, crossover, and stepped-wedge RCTs) in which the intervention being evaluated incorporates an AI or ML component—whether diagnostic imaging triage, predictive sepsis alerting, closed-loop physiological dosing, or ambient clinical decision support. Within the broader EQUATOR Network AI reporting ecosystem (see AI reporting guidelines across CONSORT-AI, TRIPOD+AI, and DECIDE-AI), CONSORT-AI governs completed randomized efficacy trials, pairing directly with SPIRIT-AI (which governs clinical trial protocols before enrollment begins), DECIDE-AI (which governs early-stage, live human-factors clinical evaluations), and TRIPOD+AI (which governs diagnostic and prognostic prediction model development and validation studies).
What Are the 14 CONSORT-AI Checklist Items for Reporting AI Interventions?
Authors submitting a clinical AI randomized trial manuscript to journals endorsing the EQUATOR Network must report all standard CONSORT 2010 items plus the 14 CONSORT-AI items detailed in the table below.
| CONSORT Section & Item | Extension Type | Official CONSORT-AI Reporting Requirement | Operational Verification for Trial Authors |
|---|---|---|---|
| Title & Abstract: 1a,b (i) | Elaboration | Indicate that the intervention involves artificial intelligence / machine learning in the title and/or abstract and specify the type of model. | State the exact ML architecture (e.g., convolutional neural network, gradient-boosted tree) rather than generic terms like “smart algorithm.” |
| Title & Abstract: 1a,b (ii) | Elaboration | State the intended use of the AI intervention within the trial in the title and/or abstract. | Clarify both the target disease/condition and the clinical task (e.g., screening, triage, diagnosis, treatment recommendation). |
| Background: 2a (i) | Extension | Explain the intended use of the AI intervention in the context of the clinical pathway, including its purpose and intended users. | Specify where in the care workflow the AI sits and who acts on its output (e.g., radiologists, triage nurses, primary care physicians, or patients). |
| Participants: 4a (i) | Elaboration | State the inclusion and exclusion criteria at the participant level. | Distinguish patient demographic/clinical eligibility from technical data-input eligibility. |
| Participants: 4a (ii) | Extension | State the inclusion and exclusion criteria at the input data level. | Specify minimum image resolution, required DICOM tags, laboratory assay types, or wearable sampling frequencies required for a record to enter the AI pipeline. |
| Interventions: 5 (i) | Extension | State which version of the AI algorithm was used. | Report the exact software version identifier, model checkpoint hash, and whether the model was locked throughout the trial or continuously updated. |
| Interventions: 5 (ii) | Extension | Describe how the input data were acquired and selected for the AI intervention. | Detail hardware scanner manufacturers/models, preprocessing pipelines, normalization, and whether a human manually cropped or selected input frames. |
| Interventions: 5 (iii) | Extension | Describe how poor-quality or unavailable input data were assessed and handled. | State the automated or manual quality threshold, how many inputs failed quality checks, and the fallback clinical pathway when the AI refused an input. |
| Interventions: 5 (iv) | Extension | Specify whether there was human–AI interaction in the handling of the input data, and what level of expertise was required of users. | Document user onboarding, mandatory training curricula, interface ergonomics, and whether users assisted in feature extraction or region-of-interest selection. |
| Interventions: 5 (v) | Extension | Specify the output of the AI intervention and how it contributed to decision-making or other elements of clinical practice. | Describe the exact format of the output (e.g., binary alert, calibrated probability, segmentation saliency map) and whether clinicians could override it. |
| Interventions: 5 (vi) | Extension | Explain the procedure for how the AI intervention was integrated into the trial setting, including any onsite or offsite requirements. | Specify local GPU vs. cloud inference latency, EHR/PACS integration architecture, and network dependency requirements at participating sites. |
| Harms: 19 (i) | Extension | Describe results of any analysis of performance errors and how errors were identified, where applicable. If no such analysis was planned or done, justify why not. | Report subgroup false-positive/false-negative rates, distribution-shift failures, and clinical consequences of erroneous AI recommendations. |
| Other Info: 25 (i) | Extension | State whether and how the AI intervention and/or its code can be accessed or reused, including any restrictions to access or re-use. | Provide repository URLs, container identifiers, licensing terms, or commercial intellectual-property restrictions preventing code release. |
Note: Because item 1a,b (i) and 1a,b (ii) each apply across both Title (1a) and Abstract (1b), the CONSORT-AI Steering Group enumerates these 13 distinct row specifications as 14 total checklist items (3 elaborations and 11 extensions).
How Does CONSORT-AI Require Authors to Report Algorithmic Failure Cases and Safety Harms?
Two of the most rigorous contributions of CONSORT-AI are Item 5 (iii) (handling of poor-quality or unavailable input data) and Item 19 (i) (analysis of algorithmic performance errors). In retrospective machine learning benchmarks, investigators routinely exclude motion-blurred radiographs, hemolyzed blood samples, or incomplete EHR records before computing area under the receiver operating characteristic curve (AUROC). If those same ungradable inputs are silently excluded from the denominator of a prospective clinical trial, the reported trial efficacy will artificially inflate real-world performance.
Under CONSORT-AI Item 5 (iii), trialists must explicitly report the volume of poor-quality or missing inputs in the CONSORT participant flow diagram and specify what clinical pathway those patients received when the AI system failed to generate an output. Under Item 19 (i), standard adverse event reporting is supplemented by mandatory algorithmic error analysis: investigators must examine cases where the AI output was discordant with the reference standard or led to inappropriate clinical escalation or delayed care, stratify error modes across demographic and hardware subgroups, and document whether clinician override mitigated or compounded the algorithmic error.
Frequently Asked Questions
What is the CONSORT-AI extension and what types of clinical AI trials must follow it?
CONSORT-AI is the 2020 extension to the CONSORT 2010 reporting statement designed for randomized controlled trials that evaluate clinical interventions incorporating an artificial intelligence or machine learning component, regardless of whether the AI task is diagnostic, prognostic, or therapeutic.
What are the 14 CONSORT-AI checklist items for reporting AI intervention versions, input data quality, and human-AI workflow integration?
The 14 CONSORT-AI items require authors to specify the AI model type and intended use in the title/abstract (1a/b), clinical pathway context (2a), dual participant-level and input-data-level eligibility (4a), algorithm version (5i), input data acquisition (5ii), poor-quality data handling (5iii), human-AI interaction and user training (5iv), output format and decision impact (5v), site infrastructure integration (5vi), algorithmic error analysis (19i), and code/model accessibility (25i).
How does CONSORT-AI require authors to report algorithmic failure cases and safety harms?
CONSORT-AI Item 19(i) requires trial authors to report any systematic analysis of algorithmic performance errors—including how false-positive, false-negative, or out-of-distribution failures were identified and their downstream clinical consequences—or explicitly justify why an error analysis was not performed.
