Variance Inflation Factor and Multicollinearity: What It Actually Breaks

Multicollinearity is one of the most over-treated problems in applied regression. Predictors correlate, a variance inflation factor comes back above 10, and a variable gets dropped — often the one the study was about. Most of that is unnecessary, because multicollinearity damages a narrower thing than its reputation suggests.

What VIF is

For each predictor, regress it on all the other predictors and take the resulting R². The variance inflation factor is:

VIF_j = 1 / (1 − R_j²)

where R_j² is the proportion of variance in predictor j explained by the others. So VIF is not a mysterious diagnostic — it is a restatement of how predictable one predictor is from the rest. A VIF of 1 means a predictor is orthogonal to the others; a VIF of 10 corresponds to R² = 0.9 for that predictor against the rest.

Tolerance, reported by some packages, is simply the reciprocal: 1/VIF.

What multicollinearity actually breaks — and what it does not

This is the part that gets inverted in practice.

It does not bias your coefficients. Strong collinearity between an exposure and measured covariates does not introduce bias; it affects only the precision of the estimate, by inflating its standard error. The estimates remain unbiased — they are just noisier.

It does not harm prediction. Multicollinearity does not affect the model’s overall fit or its predictions. If your goal is forecasting rather than interpreting individual effects, high VIFs may be irrelevant to your purpose.

What it does break is inference about individual coefficients. Inflated standard errors widen confidence intervals and destabilise individual estimates — which is how you get coefficients that swing in magnitude, or flip sign, when a correlated predictor is added or removed. That instability is the real symptom, and it is a precision problem, not a bias problem.

The practical consequence: “my VIF is 12” is not by itself a reason to change anything. “My coefficient of interest has a confidence interval so wide it is uninformative, and it moves substantially when I add a correlated covariate” is.

The thresholds are conventions, not tests

The familiar cut-offs are VIF above 5, or above 10, as points at which collinearity “might be problematic”. They are useful conventions and they are widely reported. They are also arbitrary.

The published criticism is direct: such thresholds may be arbitrary, some contexts tolerate VIFs far above 10, and in other contexts a VIF of 10 already represents extreme collinearity. The recommended alternative is to identify the cause of the high VIF and make a contextually informed judgement about the model, rather than applying a fixed number.

That reframing matters because the causes have different implications:

• Structural collinearity you created. Interaction terms and polynomial terms are correlated with their components by construction. This is arithmetic, not a data problem, and centring the variables usually removes it.
• Two measures of the same construct. Height and weight, or three related severity scales. The model cannot separate them because they are not conceptually separate. The fix is a modelling decision — pick one, or combine them — not a diagnostic threshold.
• A variable that is nearly a function of others. Including all categories of a factor plus a total, for example. This is a specification error and should be fixed as one.
• Genuine correlation in the world. Age and years of exposure really do move together. Here high VIF is information, not a defect: your data cannot separate those effects, and no diagnostic will change that.

What to do about it

1. Ask what the model is for. Prediction: usually ignore it. Interpreting one coefficient: only collinearity involving that predictor matters. A high VIF on a nuisance covariate you never interpret is not a problem.
2. Centre interaction and polynomial terms before concluding anything, since that removes collinearity you introduced yourself.
3. Check for a specification error — a redundant total, a fully-collinear set of dummies.
4. If two variables measure one construct, decide which question you are asking and model accordingly. Combining them into an index is a legitimate answer; so is keeping one.
5. Do not drop your exposure of interest because it correlates with a confounder. Removing a confounder to reduce VIF reintroduces confounding bias — trading a precision problem for a validity one, which is the worse trade.
6. Consider penalised regression where prediction under collinearity is the goal, accepting that the coefficients then become harder to interpret individually.

Related

Before computing VIFs, it is usually worth simply looking at the relationships: see how to read a correlation matrix. Where collinearity is genuine and prediction is the goal, ridge and lasso regression handle it by shrinking coefficients, and principal component analysis replaces correlated predictors with orthogonal components at the cost of interpretability. For the related and more surprising case where adding a correlated variable changes another’s sign, see suppressor variables.

Frequently asked questions

How is VIF calculated?

Regress each predictor on all the others, take that R², and compute 1/(1−R²). A VIF of 10 corresponds to an R² of 0.9 for that predictor against the rest.

Is a VIF above 10 a problem?

Not automatically. Above 5 or above 10 are common conventions, but they are arbitrary: some contexts tolerate much higher values and in others 10 is already extreme. Diagnose the cause rather than applying the number.

Does multicollinearity bias my results?

No. It inflates standard errors and therefore reduces precision. The coefficients remain unbiased; they are just estimated less precisely and become unstable across specifications.

Does it hurt predictive accuracy?

No. Overall fit and predictions are unaffected. If you only need predictions, high VIFs may not matter at all.

Should I drop a variable with a high VIF?

Only after asking why it is high. Never drop a confounder purely to lower VIF — that exchanges an imprecision problem for a bias problem.

Why does my interaction term have a huge VIF?

Because it is built from its components and correlates with them by construction. Centre the variables first; the collinearity usually disappears and is not evidence of anything about your data.

References

• Multicollinearity in regression analyses conducted in epidemiologic studies — PMC4888898
• Multicollinearity and misleading statistical results — PMC6900425
• Regression diagnostics — UCLA OARC