Written and maintained by CASRAI Editorial Board
Last updated
Multiple imputation (MI) fails for a specific, avoidable reason far more often than for an exotic statistical one: the imputation model was weaker than the analysis it was meant to support. The mechanics of drawing several completed datasets and pooling them are well documented — see this cluster’s companion guides on full information maximum likelihood as the no-imputation alternative, Rubin’s rules for the pooling formulas themselves, and tipping-point analysis for stress-testing the result afterward. What those guides assume you already have is m properly completed datasets. This guide is the step in between: the practical protocol for actually producing them — choosing which variables belong in the imputation model and why, matching each variable’s imputation method to its data type, deciding how many imputations is enough, and checking the output before you trust it. Get this step wrong and no amount of correct pooling downstream fixes it; Rubin’s rules pool whatever you feed them, correctly or not.
Step 1: Confirm the Assumption Before You Start
MI is valid under the same assumption FIML depends on: the data are missing at random (MAR) — the probability a value is missing may depend on other observed variables, but not on the missing value itself after conditioning on those observed variables. Neither MI nor FIML can verify MAR from the data alone; it’s an assumption you argue for using subject-matter knowledge about why values are missing (a mechanism more often driven by non-response, attrition, or a broader selection process than by the missing value itself). If that argument doesn’t hold — missingness plausibly depends on the unobserved value even after conditioning on everything measured — you have a missing-not-at-random (MNAR) problem that neither MI nor FIML solves by default; that requires an explicit selection or pattern-mixture model, or at minimum the sensitivity analysis in Step 7.
Assuming MAR holds, MI and FIML are both consistent estimators, so the choice between them is practical, not statistical: MI is the better fit when you need to run several different analyses off the same completed data, when your variables mix data types badly suited to a single joint-normal likelihood, or when your downstream software doesn’t support FIML for the model you’re fitting. FIML is simpler when a single model estimates everything and your software supports it directly. The FIML guide linked above works through that trade-off in more detail; this guide assumes you’ve already landed on MI.
Step 2: Build the Imputation Model — Add Auxiliary Variables
An auxiliary variable is one that isn’t in your analysis model at all but earns a place in the imputation model anyway. The inclusive strategy — deliberately imputing with more variables than the analysis model uses — is the standard recommendation in the missing-data methodology literature over a purely “restrictive” imputation model that mirrors the analysis model exactly. A good auxiliary variable does one of two things: it correlates with the variable that has missing values (recovering information that would otherwise be lost), or it correlates with the probability of being missing in the first place (making the MAR assumption more plausible by explaining more of why data are absent). The best auxiliary variables do both.
The reason this matters is congeniality: the imputation model has to be at least as rich as the analysis model, or the completed data can’t validly support that analysis. Concretely, your imputation model should always include: every variable in the analysis model (predictors, outcome, and any interaction or transformed terms computed from them — imputing then computing the interaction is usually wrong; compute the interaction as its own passively-imputed term), plus any variable that plausibly drives the missingness mechanism even if the analysis never uses it.
A reproducible illustration: what omitting one auxiliary variable costs
The simulation below is fully synthetic and seeded for reproducibility — it exists to isolate one mechanism, not to represent any real dataset. 500 replications, each with n=400. In every replication: X and Z are independent standard normal variables; the true outcome model is Y = 2 + 0.5·X + 1.0·Z + ε (ε ~ N(0,1)); the analysis of interest regresses Y on X alone (Z is a real driver of Y but isn’t a variable the analysis cares about — a common situation, e.g. a measured covariate that’s a nuisance to the question being asked rather than part of it). Missingness in Y is generated as MAR given both X and Z (logit P(missing) = −0.3 + 0.8·X + 0.9·Z), averaging 44.2% missing on Y across replications — deliberately high, to make the effect visible in a moderate sample without requiring an enormous replication count.
Three approaches to the same 500 datasets, all pooled the same way (Rubin’s rules) where applicable:
- Complete-case analysis (drop rows missing Y): mean estimated coefficient on X = 0.3815 against a true value of 0.5 — bias −0.1185, RMSE 0.1529.
- MI with an imputation model of Y ~ X only (Z omitted, matching the analysis model exactly — the “restrictive” strategy): mean pooled coefficient = 0.3825, bias −0.1175, 95% CI coverage of the true value only 77.6% (nominal is 95%). Running MI did not fix the bias, because the imputation model was missing the variable that actually explains why data are missing.
- MI with an imputation model of Y ~ X + Z (Z included as an auxiliary variable, then dropped from the final analysis model): mean pooled coefficient = 0.4983, bias −0.0017, 95% CI coverage 94.6% — correctly calibrated, because the imputation model now matches the true missingness mechanism even though the analysis model never uses Z.
The middle result is the one worth sitting with: running MI with an inadequate imputation model performed no better than doing nothing and dropping the incomplete cases. Multiple imputation is not a fix that works by default just because you ran it — it’s only as good as the model behind the fill-in step.
Step 3: Match the Imputation Model to Each Variable’s Data Type
Every incomplete variable gets its own conditional imputation model (this is what “chained equations” / MICE means — cycling through each incomplete variable in turn, imputing it conditional on all the others, and repeating). Using the wrong model family for a variable’s type produces implausible imputed values even when the rest of the protocol is correct:
- Continuous, roughly normal: Bayesian linear regression (mice calls this method
norm) — draws imputation-model parameters from their posterior, then draws imputed values with residual noise, so uncertainty about the missingness is reflected in the spread of the m completed datasets, not just around a single point prediction. - Continuous but skewed, bounded, or with a natural floor (cost, duration, count-like measures): predictive mean matching (PMM) instead of
norm. PMM finds the closest-matching donors among the observed cases with similar predicted values and imputes an actually-observed value from among them, so imputed values can’t fall outside the observed range or take on physically impossible values (a negative duration, for instance) the way a linear-regression draw can. - Binary: logistic regression.
- Unordered categorical (more than two levels, no natural order): polytomous (multinomial) logistic regression.
- Ordered categorical (a Likert item, a staging variable): proportional-odds / ordinal logistic regression, not treated as continuous.
- Count data: Poisson or negative-binomial imputation, or PMM if the count distribution is irregular.
- A variable computed from other variables (a derived total, an interaction term, a ratio): passive imputation — compute it from the already-imputed components on each completed dataset rather than imputing it directly as though it were independently missing.
Mean/mode single imputation and last-observation-carried-forward are not part of this protocol at all: both understate uncertainty by construction (every imputed value is identical or trivially extrapolated, so the between-imputation variance Rubin’s rules relies on to widen the standard error correctly is either zero or fabricated) and, for LOCF specifically, assume no meaningful change over the missing period — an assumption that is rarely defensible and rarely stated as one when the method is used anyway.
Step 4: Decide How Many Imputations
The old rule of thumb — three to five imputations — predates the computing-cost concerns that motivated it and is now considered too low for most applications. The current standard guidance, from Stef van Buuren’s Flexible Imputation of Missing Data (the reference text behind the R mice package’s own defaults, fetched directly for this guide): the number of imputations should be similar to the percentage of cases that are incomplete — roughly m ≈ 30 if 30% of cases have missing data on some variable. That source also ties the choice more precisely to γ₀, the fraction of missing information (FMI, the same quantity Rubin’s rules computes as a byproduct of pooling):
| Fraction of missing information (γ₀) | Recommended m |
|---|---|
| 0.1 | 20 |
| 0.3 | 20 |
| 0.5 | 40 |
| 0.7 | 100 |
Because FMI is only known after you’ve imputed and pooled once, the practical workflow is iterative: impute with a provisional m (5 is fine for building and checking the imputation model itself, per the same source), inspect the FMI Rubin’s rules reports, and re-run with a larger m if the table above calls for one before treating the result as final.
A second, single-dataset simulation shows why that check matters: n=600, missingness generated with a higher intercept in the same logit mechanism as Step 2’s illustration, giving 61.5% of cases missing on Y. Imputing with the correctly-specified model (Y ~ X + Z, m=20 — the same m used throughout the Step 2 illustration) and pooling gives FMI = 0.444. Read against the table above, an FMI that close to the 0.5 row calls for roughly m=40, not the m=20 that was actually used — a reminder that a single fixed default m, however reasonable it looked going in, can turn out to be too low once the real FMI of a specific analysis is known, and the fix is simply to re-impute with more.
Step 5: Check the Imputations Before You Trust Them
Three checks, in order of how often they catch a real problem:
- Plausibility: plot the distribution of imputed values against the distribution of observed values for each incomplete variable. Imputed values that cluster somewhere the observed data never go (negative ages, a probability outside 0-1, a Likert response between categories) mean the imputation model for that variable is wrong — usually the data-type mismatches in Step 3, most often
normused on a variable that needed PMM or a bounded, discrete method instead. - Convergence: for chained-equations imputation, trace the mean and standard deviation of each imputed variable across MICE iterations. They should mix around a stable level with no drift or trend by the last few iterations; a persistent trend means the chain hasn’t converged and the imputations from an early iteration aren’t yet drawing from a stable conditional distribution.
- Congeniality, re-checked: confirm every variable, interaction, and transformation used in the analysis model actually appears in (or is passively imputed from) the imputation model, per Step 2. This is the single most common protocol failure and the one the Step 2 simulation was built to make visible — it produces no error message, no convergence warning, and no obviously implausible imputed value; the numbers look fine and the bias survives anyway.
Step 6: Pool With Rubin’s Rules
Once you have m completed, checked datasets, fit the actual analysis model separately on each and combine the m point estimates and m standard errors into one pooled result. The formulas, the within/between-imputation variance decomposition, the small-sample degrees-of-freedom adjustment, and a fully worked hand example are covered in Rubin’s Rules for Pooling Multiply Imputed Estimates — this protocol guide won’t re-derive them. One thing worth flagging here rather than there: pooling also reports FMI for each parameter, which is the number to check against the table in Step 4 to confirm your chosen m was actually large enough for this specific analysis, not just for the missingness rate in general.
Step 7: Report It the Way a Reviewer Can Check
A methods section describing MI should state, at minimum: the percentage of missing data per variable that was imputed; the assumed mechanism (MAR) and the reasoning for it; every variable in the imputation model, distinguishing analysis-model variables from auxiliary ones; the imputation method used per variable (or per data type, if uniform); the number of imputations (m) and, ideally, the FMI that justified it; the software and package/version used; and the pooling method (Rubin’s rules, almost always). If there’s real uncertainty about whether MAR holds, report a sensitivity analysis alongside the primary MI result — tipping-point analysis is the standard way to show how large a departure from MAR would have to be before the conclusion changes.
Common Mistakes
- Restrictive imputation model. Building the imputation model to mirror the analysis model exactly, with no auxiliary variables — see Step 2’s simulation for what this actually costs.
- Imputing after splitting into training and test sets for a predictive-modeling use case, letting information leak from test rows into the imputation model fit on the full data. Fit the imputation model on the training data only, then apply it (not re-fit it) to the test set.
- Treating m as a one-time decision. Choosing m=5 or m=20 up front and never checking the FMI that pooling actually produces, per Step 4.
- Imputing an outcome that will be excluded from the analysis anyway. If a case’s outcome is missing and the case will be dropped from that specific model regardless (e.g. it’s excluded for a different, real eligibility reason), imputing that outcome for the sole purpose of an analysis that will never use it just adds noise.
- Mean or single imputation as a shortcut. Understates uncertainty by construction, as noted in Step 3 — it is not a faster version of MI, it is a different and generally invalid method.
- Ignoring convergence and plausibility diagnostics because the software ran without errors. Absence of an error is not evidence the imputation model was adequate.
Practical Checklist
- Argued for MAR (or decided MI/FIML isn’t the right tool and you need an explicit MNAR model instead).
- Imputation model includes every analysis-model variable, interaction, and transformation, plus genuine auxiliary variables correlated with the missing variable and/or the missingness mechanism.
- Each incomplete variable’s imputation method matches its data type (Step 3), not a single default method applied to everything.
- m chosen provisionally, then re-checked against the FMI pooling actually reports, per the table in Step 4.
- Imputed values checked for plausibility against the observed distribution; chained-equations convergence checked.
- Pooled with Rubin’s rules; FMI recorded.
- Methods section reports missingness rate, mechanism assumption, imputation-model variables, method per data type, m, software, and pooling method.
Frequently Asked Questions
Is five imputations enough?
Not as a final answer. Five is a reasonable number while you’re still building and debugging the imputation model itself, but the number that should appear in a reported result is one checked against the FMI that pooling actually produces for your parameters of interest, per Step 4 — often 20-100 for moderate-to-heavy missingness, not 3-5.
Do I include the outcome variable in the imputation model?
Yes, if it has missing values that will be imputed, and yes as a predictor for imputing other variables even where its own values are observed — omitting the outcome from the imputation model biases the imputed predictor values toward the null association with that outcome. The exception is the “impute the outcome then immediately drop those rows” pattern in the pitfalls above, which is a different mistake, not a reason to exclude the outcome generally.
What if a candidate auxiliary variable has missing data of its own?
It’s imputed too, as just another variable in the same chained-equations cycle — auxiliary variables don’t need to be complete to be useful, they need to be genuinely correlated with the missingness mechanism and/or the incomplete variable of interest.
Should I impute separately by subgroup (e.g. by study site or treatment arm)?
Only if the relationships between variables genuinely differ by subgroup and the subgroup itself has no missing data. Where that’s plausible, the more common approach is to include the subgroup indicator (and its interactions with other imputation-model variables) directly in a single imputation model rather than running fully separate imputations, which fragments the sample used to estimate each imputation model and can make FMI worse rather than better.
Is MICE the same thing as multiple imputation?
MICE (multiple imputation by chained equations) is one way to produce the m completed datasets — the variable-by-variable, conditional-model approach used throughout this guide. Joint modelling (drawing all incomplete variables at once from one multivariate distribution, typically multivariate normal) is the other main approach and is more restrictive about mixed data types but doesn’t require the iterative convergence check in Step 5. Both, once you have m completed datasets, are pooled identically with Rubin’s rules.
