Love Plots: Reading Covariate Balance After Matching or Weighting

A love plot is the standard graphic for reporting covariate balance before and after propensity-score matching or weighting: one point per covariate, plotted on a common standardized-mean-difference (SMD) scale, usually shown twice per covariate — once for the unadjusted sample, once for the matched or weighted sample — so a reader can see at a glance which covariates started off imbalanced and whether the adjustment actually fixed them. It is named for Thomas E. Love, the biostatistician who popularized this dot-plot format for balance reporting, and it is now built into the standard causal-inference toolchains researchers actually use — the R package cobalt ships a love.plot() function specifically for this, and it is the figure most methodologists expect to see in a paper that reports propensity score matching or weighting.

This guide covers how to read one, the <0.1 standardized-mean-difference convention that governs how a love plot gets interpreted, the companion variance-ratio check that a bare SMD comparison misses, and — using a reproducible simulation, not an invented example — what it actually means when a covariate’s point does not cross the 0.1 line after adjustment.

What a love plot shows, and why it replaced the balance table

Before love plots were standard, balance was usually reported as a table: one row per covariate, one column for the unmatched SMD, one column for the matched SMD. That table is still useful and still appears in most papers — CASRAI’s own propensity score matching guide works through one. A love plot is the same information turned into a graphic: covariates run down the y-axis (often sorted by unadjusted imbalance, worst at the top), the SMD runs along the x-axis, and each covariate gets two points — one for “before,” one for “after” — connected or distinguished by shape or color. A vertical reference line, almost always at 0.1, marks the conventional threshold for adequate balance.

The reason the plot form caught on where the table didn’t disappear: a table with fifteen or twenty covariates (routine in a real propensity model) is tedious to scan for the pattern that actually matters — did adjustment pull every point inside the threshold, or did one or two covariates stay stubbornly outside it? A love plot makes that pattern visually obvious in a way a column of decimals does not. It is also, not incidentally, harder to bury a bad result in: a paper that reports a balance table with one inconveniently large post-matching SMD can be skimmed past; a love plot with one point sitting visibly to the right of the threshold line is difficult to miss.

The <0.1 SMD threshold convention

Standardized mean difference expresses the difference in a covariate’s mean between the treated and comparison groups in units of pooled standard deviation — the same construction as Cohen’s d, applied to a baseline covariate instead of an outcome. It does not shrink automatically with sample size the way a t-test p-value does, which is exactly why SMD, not a significance test on baseline covariates, is the field’s standard balance metric.

The convention most commonly cited — and the one CASRAI’s propensity-score-matching guide already uses — treats an absolute SMD under 0.1 as negligible imbalance, most often traced to Austin (2009, 2011) in the medical and health-services literature. Some fields use a more lenient 0.25 as “acceptable,” and a few use 0.2. None of these is a statistical law with a derivation behind it the way a confidence-interval formula is; they are conventions, and a rigorous paper states which one it is using rather than leaving 0.1 unstated as if it were the only possible choice. What is not conventional, though, is the underlying logic: SMD should not depend on the adjustment method used to compute the groups being compared, so the same 0.1 line applies whether the “after” point in a love plot comes from matched pairs or from inverse-probability weights.

The variance-ratio check a love plot’s mean-based axis misses

SMD compares means. Two groups can have identical, well-balanced means on a covariate while still having very different spreads — imagine a matched sample where the treated group’s ages cluster tightly around 50 while the control group’s ages range from 30 to 70 with the same average. The love plot’s SMD axis would show that covariate as balanced, because the means agree, even though the two groups’ age distributions look nothing alike.

The standard supplementary check, proposed by Rubin (2001) alongside the standardized-difference approach, is the variance ratio: the ratio of a covariate’s variance in the treated group to its variance in the comparison group, computed after matching or weighting. A ratio near 1 indicates similar spread; Rubin’s own recommendation treats a ratio between roughly 0.5 and 2 as acceptable, with values outside that range flagging a distributional mismatch that a mean-difference metric alone will not catch. Some balance-reporting packages plot variance ratios as a second panel next to the love plot for exactly this reason — a covariate can pass the SMD test and still fail the variance-ratio test, and a full balance assessment checks both, not just the one that produces the more familiar-looking dot plot.

A reproducible worked example: matching versus weighting

The numbers below come from a synthetic dataset generated and analyzed in this session with a fixed random seed (mulberry32, seed 20260829) — not real patient or study data, and not hand-picked to illustrate a tidy conclusion. The simulation: 1,000 units, four baseline covariates (age, a standardized baseline severity score, a binary comorbidity indicator, and sex — the last one deliberately unrelated to treatment assignment, a true null covariate included as a check on the method itself). Treatment assignment depends on age, severity, and comorbidity through a known logistic model; a propensity score was then estimated by fitting logistic regression from scratch (Newton-Raphson) on those same four covariates, without using the true assignment probabilities.

Two adjustment methods were run against the fitted propensity score: 1:1 nearest-neighbor matching without replacement (caliper = 0.2 × the pooled standard deviation of the logit propensity score, the conventional caliper width, applied the same way covered in the Stata implementation guide), and inverse-probability-of-treatment weighting (ATE weights, untrimmed). Standardized mean differences and variance ratios were computed before adjustment, after matching, and after weighting:

Read as a love plot, the pattern in that table is exactly what the plot form is built to make visible at a glance: every “before” point sits well to the right, most of them past 0.6, and every “after” point — whether matched or weighted — lands close to the left edge, comfortably inside the 0.1 threshold line. The one covariate that started near zero (sex, the null covariate) is the one point that barely moves, which is the correct behavior for a covariate that was never actually related to treatment assignment.

By the numbers in the table, both methods worked: every SMD lands well under 0.1 after either matching or weighting, including the null covariate (sex), which was never imbalanced to begin with and stayed that way — a useful check that the fitted propensity model isn’t distorting a covariate it shouldn’t be touching. That is the clean part of the result, and it is genuinely clean because the propensity model here was correctly specified (the logistic regression recovered coefficients close to the data-generating values — 0.565 estimated versus 0.55 true for the standardized-age effect, 0.704 versus 0.70 true for comorbidity, and 0.006 versus 0 true for the null sex term). Real propensity models are rarely this well specified, which is precisely why the love plot is checked on real analyses instead of assumed.

The result is not uniformly tidy, though, and the two methods did not win in the same way. Matching hit its excellent balance at a real cost: of the 502 treated units, only 334 (66.5%) found a control within the caliper and were retained — 168 treated units, one in three, were dropped from the matched sample entirely because no sufficiently similar control existed for them. That changes the population the estimate describes, from “the effect for all treated units” to “the effect for the subset of treated units who had a comparable control” — a real, if often under-reported, consequence of matching well. Weighting kept the full sample (Kish effective sample size 815 out of 1,000 raw units, a smaller efficiency loss than matching’s one-in-three case loss) but leaned on some large weights to do it — the largest weight in this run was 9.36, meaning a single control unit was doing the statistical work of more than nine average-weighted units, which is exactly the kind of high-influence unit that makes untrimmed IPW estimates unstable in a real, smaller dataset even when the resulting SMDs look fine on average. Neither of those costs shows up on the love plot itself, which is one of the plot’s real limitations: it reports balance, not sample retention or weight stability, and both matter for whether the adjusted estimate is trustworthy.

One more non-tidy detail worth reporting plainly rather than smoothing over: the baseline severity score’s variance ratio came out at 1.175 after matching and 1.086 after weighting — both inside Rubin’s 0.5-2.0 acceptable range, but neither at the 1.0 that “balanced” intuitively suggests, despite that same covariate’s SMD landing at 0.001 and 0.010 respectively. That is the variance-ratio check earning its place in the analysis: a covariate can look essentially perfectly balanced on the love plot’s own axis while still carrying a detectably different spread between groups.

What an unbalanced covariate after matching actually means

None of the covariates above stayed outside the 0.1 line after adjustment, so the point below is conceptual, illustrating what the diagnostic is for rather than restating the (fully balanced) results above as if they showed something they didn’t. Suppose a love plot instead showed one covariate’s post-matching point sitting at, say, 0.18 — past the threshold. That is not a cosmetic problem to note in a limitations paragraph and move past. It means the matching or weighting procedure, as specified, did not equalize that covariate between groups, and any outcome comparison that follows is still confounded (see confounding variable) by whatever that covariate captures, to a degree the SMD itself quantifies.

The response is not to re-run the outcome model and hope the imbalance washes out. The standard next steps, roughly in the order a methodologist would try them:

• Respecify the propensity model. Add an interaction term or a nonlinear transform (a quadratic, a spline) involving the imbalanced covariate — often the covariate itself is fine, but the propensity model’s functional form doesn’t capture how it relates to treatment assignment, and a richer specification fixes exactly that covariate’s balance without touching the others.
• Tighten the matching, or switch matching methods. A narrower caliper, exact matching on the problem covariate specifically (matching it exactly rather than through the propensity score), or a different algorithm (full matching, genetic matching) can succeed where 1:1 nearest-neighbor matching left that one covariate unresolved — usually at the cost of discarding more units, the same trade-off the worked example above shows directly.
• Check for a common-support problem. An unbalanced covariate after matching is sometimes a symptom of poor overlap — some region of that covariate’s range exists almost exclusively in one group, so no good match is available there. Trimming to the region of common support (and reporting that the estimate no longer applies to the trimmed-out population) is a legitimate response, not a way of hiding a problem.
• Add the covariate as a regression adjustment on top of the match. Combining matching with regression adjustment for covariates that remain imperfectly balanced is a standard, defensible combination — sometimes described as doubly robust in spirit — though it is a patch on a specific known-imbalanced covariate, not a substitute for trying to achieve balance in the first place.
• Report it as a limitation if none of the above resolves it. If a covariate cannot be balanced with the available data, the honest response is to say so plainly next to the love plot, not to omit the covariate from the plot or round its SMD down. A love plot with one point past the line, reported as such, is more trustworthy than one that quietly excludes the covariate that didn’t cooperate.

What an unbalanced covariate on a love plot never means is that the difference “probably doesn’t matter” because it’s not statistically significant on some ad hoc test — the whole reason SMD replaced significance testing for this purpose is that significance testing answers a different question (is the sample large enough to detect this difference) than the one that matters here (is the difference large enough, in absolute terms, to plausibly bias the outcome comparison).

Frequently asked questions

Does a love plot work the same way for inverse-probability weighting as for matching?

Yes — the SMD and variance-ratio calculations are simply computed on the weighted sample (using the propensity weights to compute weighted means and weighted variances) instead of on matched pairs, and the same 0.1 threshold and love-plot format apply. The worked example above reports both side by side on the same covariates for exactly this reason: the diagnostic doesn’t change with the adjustment method, only the numbers going into it do.

Is 0.1 the only threshold anyone uses?

No. It is the most commonly cited convention, most often traced to Austin (2009, 2011), and it’s the one CASRAI’s own propensity-score-matching guide uses. Some fields treat 0.25 as an acceptable ceiling and 0.1 as ideal-but-not-mandatory. There is no single enforced statistical cutoff; a paper should state which convention it is using.

Can a covariate pass the SMD check and still be a problem?

Yes, in two ways covered above: its variance ratio can still be outside the acceptable range even with a near-zero SMD (a spread problem, not a mean problem), and passing the check on measured covariates says nothing about an unmeasured confounder that was never in the propensity model at all — balance diagnostics can only ever certify balance on the covariates that were actually measured and included.

Should every covariate in the propensity model appear on the love plot?

Generally yes — omitting a covariate from the balance report because its post-adjustment SMD is inconvenient defeats the purpose of the diagnostic. Interaction terms and polynomial terms included only to help the propensity model fit are sometimes left off a love plot meant for a general audience, but the underlying substantive covariates that readers would recognize as “the confounders” should all be shown.