Written and maintained by CASRAI Editorial Board
Last updated
Measurement bias is a systematic difference between what an instrument, procedure or respondent reports and the true value of what was meant to be measured. Because it pushes results in one direction, collecting more data does not remove it. This guide defines it, shows measurement bias examples from published studies, works through the arithmetic of estimating it, and sets out how to reduce and report it.
What is measurement bias?
Measurement bias is error that is not random: the same flaw shifts every measurement the same way, or shifts a whole group of measurements differently from another group. NIST’s engineering statistics handbook, following the international vocabulary of metrology, defines bias as “the difference between the measurement result and its unknown ‘true value’.” A balance that reads 0.45 g heavy, a survey question that nudges answers upward and a thermometer that drifts as it warms all produce measurement bias.
In epidemiology and the social sciences the same idea is usually filed under information bias, which the Catalogue of Bias describes as any systematic difference from the truth that arises in the collection, recall, recording and handling of information in a study. Treating measurement bias as the instrument-and-procedure case of information bias is common usage rather than a formal definition.
Measurement bias examples
Self-report versus a monitor
Chang and colleagues (American Journal of Industrial Medicine, 2010; 53(11):1142-1149) compared self-reported daily computer use with usage-monitoring software. For usage below 3.6 hours a day, the self-reports tended to overestimate the duration of computer use. Self-report is a measurement procedure, so its errors are a case of measurement bias.
Routinely recorded smoking status
Marston and colleagues (BMJ Open, 2014; 4(4):e004958) examined smoking status recorded in UK primary care. Routine records can misclassify people: those who quit smoking at a young age or a long time ago were more likely to be recorded as non-smokers, so recorded status can differ systematically from true status.
An uncalibrated instrument
The simplest case is a device that reads consistently high or low. The worked example below uses invented numbers to show how that offset is estimated.
Measurement bias vs random error
Random error scatters readings around their own average and shrinks as you average more readings. Measurement bias moves the average itself away from the true value and does not shrink. The practical consequence: a tight spread of readings says the instrument is precise, not that it is right. See accuracy vs precision in measurement for the full distinction.
Measurement bias vs related biases
| Term | Where the error enters | Read more |
|---|---|---|
| Measurement bias | The instrument, scale, question wording or recording procedure shifts values systematically | This page |
| Information bias | Systematic differences in collecting, recalling, recording or handling study information (the broader family) | Information bias |
| Recall bias | Participants remember past exposures differently depending on outcome or group | Recall bias |
| Observer bias | The person taking the measurement is influenced by what they expect to see | Observer bias |
Formula for estimating measurement bias
When a reference value exists, bias is estimated from repeated measurements of that reference. This is the standard reading of NIST’s definition, written out here as simple arithmetic:
bias = x̄ − R, with standard error SE = s / √n and test statistic t = bias / SE on n − 1 degrees of freedom.
- x̄ (x-bar): the mean of the n repeated readings
- R: the accepted reference value, such as a certified mass standard
- s: the sample standard deviation of the readings
- n: the number of readings
Relative bias is bias / R, often shown as a percentage.
Worked example with real arithmetic
The readings below are invented for illustration. A balance weighs a 50.00 g reference mass ten times: 50.4, 50.5, 50.3, 50.6, 50.4, 50.5, 50.4, 50.3, 50.5, 50.6 g.
- Sum = 504.5, so x̄ = 504.5 / 10 = 50.45 g.
- Bias = 50.45 − 50.00 = +0.45 g; relative bias = 0.45 / 50 = 0.9%.
- Squared deviations from 50.45 sum to 0.105, so s = √(0.105 / 9) = 0.108 g.
- SE = 0.108 / √10 = 0.0342 g, so t = 0.45 / 0.0342 = 13.17 on 9 degrees of freedom.
- With the critical t of 2.262 (two-sided, 95%), the 95% confidence interval for the bias is 0.45 ± 2.262 × 0.0342 = 0.37 to 0.53 g.
Interpretation
| Result | Reading |
|---|---|
| Confidence interval for bias excludes 0 | The instrument reads systematically off from the reference; in the example it reads high |
| Confidence interval includes 0 | No systematic offset is distinguishable from random scatter at this sample size |
| Small s but large bias | Precise but inaccurate, as in the example (s = 0.108 g, bias = 0.45 g) |
| Bias that changes with the true level | Not a constant offset; compare readings across the range of true values rather than at one point |
NIST notes that bias can often be estimated and eliminated by calibration to a reference standard, but that calibration may not guarantee correct performance in the working environment, which is why gauge studies are used to uncover remaining biases.
Assumptions and common mistakes
- The reference value is treated as known. If the reference itself is uncertain, that uncertainty belongs in the result.
- The t interval assumes the readings are roughly normally distributed and independent. Readings taken in a drifting sequence are not independent.
- A statistically significant bias is not necessarily a practically important one. Judge it against the tolerance your measurement needs.
- Averaging more readings fixes random error, not bias.
- A correlation or reliability coefficient can look excellent while a constant offset remains; see reliability vs validity.
How to reduce measurement bias
The Catalogue of Bias says close attention should be paid at the design stage, as little can be done to address incorrect data afterwards. Controls it describes, with NIST’s calibration advice, mapped to the examples above:
- Standardised methods and devices (Catalogue of Bias), with calibration to a reference standard (NIST; the balance example).
- Well-designed data collection protocols with exposures and outcomes clearly defined.
- Maintaining blinding of exposure status while outcomes are measured.
- Questions posed neutrally where interviewers collect data.
- Using medical or work records to ascertain information instead of relying on self-report (the Chang and Marston examples).
When bias cannot be removed, quantitative bias analysis estimates how much it could change the conclusion.
How to report it (APA example)
Report the reference, the number of readings, the bias with its interval and the test: “Across 10 readings of a 50.00 g reference mass, the balance read 0.45 g high on average (M = 50.45, SD = 0.11), 95% CI of the bias [0.37, 0.53], t(9) = 13.17, p < .001.”
How to calculate it in Excel, R or Python
Excel
With readings in A2:A11 and the reference in D1: bias =AVERAGE(A2:A11)-D1; standard error =STDEV.S(A2:A11)/SQRT(COUNT(A2:A11)); interval half-width =T.INV.2T(0.05,COUNT(A2:A11)-1)*(STDEV.S(A2:A11)/SQRT(COUNT(A2:A11))).
R
x <- c(50.4,50.5,50.3,50.6,50.4,50.5,50.4,50.3,50.5,50.6)
mean(x) - 50
t.test(x, mu = 50) # t = 13.17, df = 9; CI is for the mean, subtract 50 for the biasPython
import numpy as np
from scipy import stats
x = np.array([50.4,50.5,50.3,50.6,50.4,50.5,50.4,50.3,50.5,50.6])
bias = x.mean() - 50
se = x.std(ddof=1) / np.sqrt(len(x))
half = stats.t.ppf(0.975, len(x)-1) * se
print(bias, bias - half, bias + half)FAQ
What is measurement bias?
A systematic difference between measured and true values caused by the instrument, procedure or respondent, which pushes results the same way and is not reduced by larger samples.
What are examples of measurement bias?
A miscalibrated balance, a survey question worded to prompt agreement, self-reported computer use that tends to overstate low daily use, and routinely recorded smoking status that can misclassify former smokers as non-smokers.
How is measurement bias different from random error?
Random error varies unpredictably and averages out; measurement bias shifts the average and does not.
Can calibration remove it?
Often, for instrument bias, by calibrating to a reference standard, though performance in the working environment should still be checked.
Sources
- Catalogue of Bias, “Information bias”: https://catalogofbias.org/biases/information-bias/ (checked 2026-10-29).
- NIST/SEMATECH e-Handbook of Statistical Methods, “Bias”: https://www.itl.nist.gov/div898/handbook/mpc/section4/mpc45.htm (checked 2026-10-29).
- Chang et al. (2010), Am J Ind Med 53(11):1142-1149; Marston et al. (2014), BMJ Open 4(4):e004958, both described in the Catalogue of Bias entry above.
