Examples
Worked examples
- Is an instance
A published model with full training script, locked-version dependencies, random seed, and Docker image producing the reported accuracy on 5 seeds.
- Is an instance
An RL experiment with deterministic environment seeds and reported variance across 30 runs.
Counter-examples
Looks similar, but isn't
- Not an instance
A paper reporting a single accuracy with no seed disclosure.
- Not an instance
A model trained with a closed API call to an undisclosed model version.
Editorial commentary
Reproducible AI experiments are a specialisation of reproducibility" class="text-primary underline-offset-2 hover:underline" data-autolinked="true" title="Computational reproducibility — CASRAI Dictionary">computational reproducibility for ML, with distinctive challenges: stochastic training, hardware-dependent floating point, distributed-training non-determinism, and dependency on closed APIs. NeurIPS, ICML, ICLR, and others have introduced reproducibility checklists. Numerical equivalence is often replaced by 'within reported variability across seeds'.
References
- Pineau et al., 'Improving reproducibility in machine learning research' (Journal of Machine Learning Research, 2021); NeurIPS Reproducibility Checklist.
Also known as
reproducible ML experiment
Machine-readable encodings
Use in your systems
<role vocab="credit"
vocab-identifier="https://casrai.org/dictionary/"
vocab-term="Reproducible AI experiment"
vocab-term-identifier="https://casrai.org/dictionary/term/reproducible-ai-experiment" />{
"@context": "https://schema.org",
"@type": "DefinedTerm",
"name": "Reproducible AI experiment",
"identifier": "https://casrai.org/dictionary/term/reproducible-ai-experiment",
"description": "An AI experiment for which sufficient artefacts and metadata are released (data, code, seed, environment, hyperparameters, training procedure) that an independent investigator can re-run it and obtain numerically equivalent or statistically indistinguishable results.",
"inDefinedTermSet": "https://casrai.org/dictionary/domain/ai-and-ml-research-outputs/",
"url": "https://casrai.org/dictionary/term/reproducible-ai-experiment",
"sameAs": [
"reproducible ML experiment"
],
"license": "https://creativecommons.org/licenses/by/4.0/"
}
