Written and maintained by CASRAI Editorial Board
Last updated
Survey fatigue is the drop in answer quality, or the decision to stop answering, that can follow when a respondent has been asked too much. It is easy to define and harder to measure, and the experimental evidence is more mixed than most advice about it suggests. This guide separates the two things the term covers, shows which indicators to compute, works through an item-nonresponse check with the arithmetic shown, and states what three published studies actually found. For the neighbouring problem of respondents who never engage at all, see detecting careless responding.
What survey fatigue is (and is not)
Survey fatigue describes respondents becoming tired, bored or disinterested during a survey, giving less thoughtful answers or ending early (Le, Han & Palamar, 2021). The term is used for two different situations:
- Within-survey fatigue (respondent fatigue). Quality falls as a respondent moves through one questionnaire. The usual mechanism is satisficing: giving an acceptable answer with less effort rather than an optimal one. All the evidence in this guide concerns this meaning.
- Across-survey fatigue (over-surveying). A population is asked so often that participation falls. This is a participation problem and belongs with non-response bias, not with answer quality.
Fatigue is not the same as careless responding. Careless responding can start on the first item, while fatigue is defined by change over the course of the instrument.
How is survey fatigue measured?
There is no single fatigue statistic. Researchers instead compute indicators of satisficing and look at how they change with item position or questionnaire length. Roberts, Eva, Allum and Lynn (2010) distinguish weak satisficing (acquiescence, response-order effects) from strong satisficing (repeated “Don’t know” answers, non-differentiation), following Krosnick (1991). The simplest indicator has a formula:
Item nonresponse rate (%) = (m ÷ (n × k)) × 100
- m = number of “Don’t know”, refused or missing answers in the block of items being examined
- n = number of respondents who reached that block
- k = number of items in the block
- n × k = the number of answers that could have been given
Other indicators: non-differentiation (identical answers across a battery of items), acquiescence (agreeing regardless of content), midpoint preference, response-order effects (choosing the first or last category) and breakoff (quitting before the end).
Worked example: item nonresponse by position
The counts below are illustrative, not from a real study. Suppose 1,000 respondents complete a 40-item survey, and we divide the items into four blocks of 10 in the order asked. Every respondent reached every block, so n = 1,000 and k = 10, giving 10,000 possible answers per block.
| Block (items) | Missing or “Don’t know” (m) | Calculation | Rate |
|---|---|---|---|
| 1 (1-10) | 150 | 150 ÷ 10,000 | 1.5% |
| 2 (11-20) | 210 | 210 ÷ 10,000 | 2.1% |
| 3 (21-30) | 330 | 330 ÷ 10,000 | 3.3% |
| 4 (31-40) | 520 | 520 ÷ 10,000 | 5.2% |
The rate rises from 1.5% to 5.2%, an increase of 5.2 − 1.5 = 3.7 percentage points, and block 4’s rate is 5.2 ÷ 1.5 ≈ 3.5 times block 1’s. This pattern is compatible with fatigue, but it does not prove it. If the later items are harder, more sensitive or less relevant, nonresponse would rise for those reasons alone. The only clean test is to present the same items at different positions, which is the design Roberts and colleagues used.
Interpretation: what each indicator tells you
| Indicator | What the studies found | Reading guidance |
|---|---|---|
| Item nonresponse / “Don’t know” | Roberts et al.: rates of 0.25, 0.26 and 0.18 in the longest, middle and shortest interview groups; the difference was not statistically significant | A rising rate across positions is a signal worth checking, but a length difference in this rate was not established in that experiment |
| Non-differentiation | Roberts et al.: no significant length effect; education predicted it | Compare respondents of similar education before blaming length |
| Acquiescence | Roberts et al.: no significant length effect; education predicted it | See acquiescence bias for balanced scales and reverse-coding |
| Midpoint preference | Roberts et al.: the shortest-interview group chose the midpoint more often | Do not treat midpoint use as a fatigue marker on its own |
| Response-order effects (first or last category) | Roberts et al.: the longest-interview group was more likely to choose first or last categories | The indicator that moved in that experiment; examine it first |
| Breakoff and completion | Galesic and Bosnjak: a longer stated length (10, 20 or 30 minutes) meant fewer people started and completed | Report completion by condition, not only answer quality among completers |
| Speed and uniformity of later answers | Galesic and Bosnjak: later answers were faster, shorter and more uniform than earlier ones | Compare early and late items for the same respondent |
What the experiments show
Roberts, Eva, Allum and Lynn (2010)
This telephone experiment randomly assigned respondents in Germany, Hungary, Poland and Switzerland to interviews of about 60, 45 or 30 minutes, built from European Social Survey questionnaires, and analysed a 46-item well-being module asked at different points. The authors report that data quality is lower when questions come later in the interview, but find only partial support for the length hypothesis. Country effects were larger than length effects for half of the indicators. They note that the literature is inconsistent on which forms of satisficing length affects, and they recommend keeping questionnaires short.
Galesic and Bosnjak (2009)
In a web survey with a stated length of 10, 20 or 30 minutes, longer stated lengths meant fewer respondents started and completed, and answers to later questions were faster, shorter and more uniform than answers to earlier ones.
Le, Han and Palamar (2021)
Among 112,184 respondents to the National Survey on Drug Use and Health (average length about 60 minutes), 9.4% (95% CI 8.9-9.8) said the survey took too long. Respondents who skipped five or more drug items had adjusted odds of 3.26 (95% CI 2.26-4.71) of saying so, against 1.68 (1.41-2.01) for skipping one or two. Adjusted odds were 5.13 (4.31-6.11) for ages 65 and over compared with ages 12-17, and 3.99 (3.51-4.53) for trouble understanding the questions; a college degree lowered them (0.72, 0.61-0.84). The study is cross-sectional and reports an association, and “too long” counted only when respondents volunteered it.
Assumptions and common mistakes
- Confusing length with position. Late items differ in content as well as position. Only items rotated across positions separate the two.
- Treating midpoint use as fatigue. In the Roberts experiment the shortest interviews produced more midpoint answers.
- Ignoring education and country. Both predicted several indicators, in some cases more strongly than length.
- Treating respondents as independent. Answers within a respondent are repeated measures, so a test that treats 10,000 answers as 10,000 independent observations overstates certainty. Use respondent-level rates or a model with a respondent term.
- Mixing the two meanings. Falling participation across repeated surveys is not evidence of within-survey fatigue.
How to report it (APA example)
State the indicator, the blocks compared and the denominator. Using the illustrative figures above: “Item nonresponse increased across questionnaire position, from 1.5% of answers in items 1-10 to 5.2% in items 31-40 (n = 1,000; 10,000 possible answers per block).” Cite the design source in APA 7th style, for example: Roberts, C., Eva, G., Allum, N., & Lynn, P. (2010). Data quality in telephone surveys and the effect of questionnaire length: A cross-national experiment (ISER Working Paper 2010-36). University of Essex.
How to calculate it in Excel, R or Python
Each snippet assumes one row per respondent, one column per item in the order asked (items 1-40), and blanks for missing or “Don’t know” answers.
Excel, rate for block 1 (columns B to K, rows 2 to 1001):
=COUNTBLANK(B2:K1001)/(ROWS(B2:K1001)*COLUMNS(B2:K1001))R:
blocks <- split(1:40, rep(1:4, each = 10))
sapply(blocks, function(ix) mean(is.na(df[, ix])))Python (pandas):
rates = [df.iloc[:, i:i+10].isna().to_numpy().mean() for i in range(0, 40, 10)]
print(rates)The mean of the missing-value indicator over all cells equals m ÷ (n × k) when every respondent reached the block.
FAQ
How long is too long for a survey?
No universal cutoff is established. The experiments above compared 30, 45 and 60 minute telephone interviews and stated web lengths of 10, 20 and 30 minutes; both found effects that depend on the outcome measured, and longer stated web length reduced starting and completing.
Is survey fatigue the same as non-response?
No. Fatigue concerns answer quality and breakoff among people who started; non-response concerns who answers at all. Over-surveying overlaps with non-response, and non-response bias covers that side.
How do you reduce survey fatigue?
The Roberts authors recommend keeping questionnaires as short as possible. Check item wording with guidance on questionnaire design and the research questionnaire entry.
Can I detect fatigue after data collection?
Yes, by computing the indicators by item position, as in the worked example, and by comparing speed and variation early versus late.
Sources
- Roberts, C., Eva, G., Allum, N., & Lynn, P. (2010). Data quality in telephone surveys and the effect of questionnaire length: A cross-national experiment. ISER Working Paper 2010-36, University of Essex. https://www.iser.essex.ac.uk/wp-content/uploads/files/working-papers/iser/2010-36.pdf (checked 2026-10-29)
- Galesic, M., & Bosnjak, M. (2009). Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opinion Quarterly, 73(2), 349-360. https://doi.org/10.1093/poq/nfp031 (abstract checked 2026-10-29)
- Le, A., Han, B. H., & Palamar, J. J. (2021). When national drug surveys “take too long”: An examination of who is at risk for survey fatigue. Drug and Alcohol Dependence, 225, 108769. https://pmc.ncbi.nlm.nih.gov/articles/PMC8282613 (checked 2026-10-29)
