Design of Experiments (DOE) is a branch of applied statistics for planning, structuring, and analyzing controlled tests so that the effect of one or more input factors on one or more outcomes can be estimated efficiently and with a known level of confidence. In a research context, DOE is what happens before a single data point is collected: it determines which factors get varied, which are held constant, how many runs are needed, and how the resulting dataset will be structured — all decisions that directly shape the data management plan (DMP) for the resulting data.
Note on the acronym: “DOE” is also the standard abbreviation for the U.S. Department of Energy, an unrelated federal agency and research funder. This page covers the statistical methodology — Design of Experiments — not the agency. If you were looking for U.S. DOE funding or data policy, that is a separate topic.
Origins and foundational principles
Modern DOE traces to the statistician Ronald A. Fisher’s work at Rothamsted Experimental Station in England in the 1920s, formalized in his 1935 book The Design of Experiments. Fisher was working on agricultural field trials and needed a way to separate the real effect of a treatment (e.g., a fertilizer) from natural variation in soil, weather, and plot position. His solution introduced three principles that remain the foundation of experimental design today:
- Randomization — assigning treatments to experimental units by a random process, so that unmeasured (confounding) variables are distributed evenly across treatment groups rather than systematically biasing one group over another.
- Replication — running each treatment combination more than once, so that natural variability (noise) can be estimated separately from the treatment effect, and the precision of the estimate can be quantified.
- Blocking — grouping experimental units into blocks that are internally similar (e.g., the same batch of reagent, the same day, the same instrument), and applying every treatment within each block, so that block-to-block variation doesn’t get mistaken for a treatment effect.
These three principles apply whether the experiment involves agricultural plots, cell cultures, materials samples, or software A/B tests — they are about the logic of causal inference under uncertainty, not about any one discipline.
Why DOE matters for data management, specifically
DOE and research data management intersect at a point that is easy to overlook: the design of an experiment determines the shape and provenance metadata of the dataset it produces, before any data exists to manage. A dataset generated from a designed experiment needs, at minimum:
- A record of the factors, their levels, and the design type used (full factorial, fractional factorial, response surface, etc.) — without this, the dataset cannot be correctly reanalyzed or reproduced by anyone other than the original researcher.
- An explicit randomization scheme or seed, if randomization was used, so that the assignment of units to treatments is auditable.
- Block identifiers as a variable in the dataset itself, not just noted in a lab notebook, since blocking is only usable in analysis if it’s captured as structured data.
- A clear run/replicate identifier distinguishing true replication (independent repeats of the whole procedure) from pseudoreplication (repeated measurements of the same run), a distinction that materially affects what statistical claims the data can support.
This is why funder and institutional DMP templates that ask researchers to describe “how data will be generated” are, in practice, asking about the experimental design as much as the instrumentation — see CASRAI’s guide to types of research data for how design choices feed into the taxonomy a DMP has to address, and the Data Management Plan (DMP) entry for what a DMP is generally expected to cover.
Common DOE design types
Full factorial design
A full factorial design tests every possible combination of every factor at every level. With k factors each at 2 levels, that’s 2k runs. Full factorial designs let you estimate every main effect and every interaction between factors, but the number of runs grows exponentially as factors are added, which quickly becomes impractical past a handful of factors.
Fractional factorial design
A fractional factorial design runs only a carefully chosen subset of the full factorial’s combinations, deliberately trading the ability to separately estimate some higher-order interactions (which are often negligible in practice) for a much smaller number of runs. These are commonly used as screening designs — an early-stage experiment to identify which of many candidate factors actually matter, before committing resources to a more detailed study of the important ones.
Response surface methodology (RSM)
Response surface methodology is used once the important factors are known and the goal shifts to optimization — finding the factor levels that maximize or minimize a response. RSM designs, such as the central composite design, add axial and center-point runs to a factorial base so that curvature in the relationship between factors and response can be modeled, not just a linear approximation.
Randomized block and Latin square designs
These designs apply the blocking principle directly: a randomized block design assigns every treatment once within each block; a Latin square design extends this to control for two blocking variables simultaneously (e.g., row and column position) using a grid where every treatment appears exactly once in every row and every column.
Taguchi methods
Developed by Genichi Taguchi for industrial quality engineering, Taguchi designs use a specific, restricted set of orthogonal arrays intended to make experiments robust to “noise” factors that cannot be controlled in production (temperature drift, material variability). They are a variant of fractional factorial design with an added emphasis on minimizing sensitivity to uncontrolled variation, not just estimating effects.
DOE and reproducibility
An experiment’s design is part of what makes it reproducible or not. A dataset without a documented design — what was varied, what was held constant, how units were assigned to conditions — cannot be meaningfully reanalyzed or replicated even if the raw numbers themselves survive. CASRAI’s reproducibility entry and its related reproducibility crisis entry cover this from the analysis side; DOE is the upstream discipline that determines whether the data even has the structure needed to support a reproducible analysis in the first place. For the infrastructure side of preserving that structure alongside the data (workflows, containers, code), see CASRAI’s guide to reproducibility infrastructure.
DOE in clinical and biomedical research
In clinical research, the most familiar application of DOE’s principles is the randomized controlled trial (RCT), which applies randomization (and often blocking, via stratified randomization) to compare an intervention against a control. Classical industrial DOE (full factorial, fractional factorial, RSM) is less commonly used directly in clinical trial design, which has its own specialized design vocabulary (parallel-group, crossover, factorial trial designs, adaptive designs), but the underlying logic — randomize, replicate, block — is the same statistical inheritance from Fisher’s original work.
Software and tools
DOE is rarely done by hand past the simplest designs. Statistical software packages with dedicated DOE modules include commercial tools such as JMP and Minitab, and open-source options such as R’s DoE.base, FrF2, and rsm packages, and Python’s pyDOE2. These tools generate the run matrix (which factor-level combinations to actually test, in what randomized order) and, after data collection, fit the statistical model used to estimate effects.
Frequently asked questions
What is the difference between DOE and one-factor-at-a-time (OFAT) experimentation?
OFAT varies a single factor while holding all others constant, then repeats for each factor in turn. It cannot detect interactions between factors (where the effect of one factor depends on the level of another) and generally needs more runs than a well-chosen factorial design to reach the same precision. DOE varies multiple factors simultaneously in a structured way specifically so that interactions can be estimated and runs used efficiently.
How many runs does a DOE experiment need?
It depends on the number of factors, the design type, and the desired power to detect an effect of a given size. Fractional factorial and screening designs exist specifically to make experiments with many candidate factors practical with a manageable number of runs; the exact number is a calculation the DOE software or a statistician performs based on the study’s specific factors and precision requirements, not a fixed rule of thumb.
Is DOE the same as A/B testing?
A/B testing is a special case of DOE with a single factor at two levels (A and B) and one primary response. The same principles of randomization and adequate replication (sample size) apply, but full DOE typically refers to designs with multiple factors and more complex structures such as fractional factorial or response-surface designs.
Do I need to describe my experimental design in my data management plan?
Most funder DMP templates ask, in some form, how data will be generated and organized. For data from a designed experiment, this should include the design type, the factors and levels, the randomization approach, and how runs/replicates/blocks are represented as variables in the dataset — not just a description of the instrument or assay used to produce the raw measurements.
