Skip to main content
v2026.11,986 entries · CC-BY 4.0

Practice Effect: Definition, Size and How to Control It

Practice effect (retest effect): score gains from repeated testing, how large they are, a worked example, and designs and adjustments that control them.

Written and maintained by CASRAI Editorial Board

Published

On this page

A practice effect is an improvement in test scores that comes from having taken the same or a similar test before, rather than from a real change in the ability, attitude or condition being measured. It is also called a retest effect. It matters in any study that measures the same people more than once, because a gain between time 1 and time 2 can be partly or wholly practice, and a change score cannot tell the two apart on its own.

What is a practice effect? Definition and synonyms

Duff and Hammers (2020) define practice effects as “improvements in cognitive test scores due to repeated exposure to testing materials.” Scharfen, Blum and Holling (2018) use “retest effects” as their main term, treat “practice effects” as a synonym, and describe them as the gain in test performance that results from repeated administration of the same or an equivalent test.

Why repeated testing raises scores

Scharfen and colleagues, following Lievens and colleagues (2007), separate three routes to a retest gain. The first is a true increase in the latent ability being tested. The second is a reduction in construct-irrelevant factors such as test anxiety or unfamiliarity with the format. The third is the acquisition of test-specific skills or strategies, including automatization of the task. Only the first of these is a change in what you meant to measure; the other two inflate the score without changing the construct.

Practice effect vs maturation vs carryover

These three are often confused because each produces a change between two time points.

  • Practice effect: the change is caused by earlier exposure to the measurement itself.
  • Maturation: participants change naturally over time (growth, ageing, recovery) whether or not they were tested before. See maturation as a threat to internal validity.
  • Carryover: in a crossover design, the effect of an earlier treatment period persists into a later one. See crossover trial design and carryover.

A practice effect can be present with no maturation and no carryover, and vice versa. A control group that is tested on the same schedule is the usual way to separate them from a treatment effect.

Practice effect formula and notation

There is no single standard formula, because a practice effect is a descriptive difference between two administrations. The simplest form is:

PE = M2 − M1

Standardised so it can be compared across tests:

dPE = (M2 − M1) / SD1

  • M1: mean score at the first administration.
  • M2: mean score at the second administration of the same or an equivalent form.
  • SD1: standard deviation of the first-administration scores (some authors use a pooled or normative SD; state which one you used).

When a practice effect is known in advance, an individual’s practice-adjusted change is the observed change minus the expected practice effect:

Adjusted change = (X2 − X1) − PEexpected

Duff and Hammers describe the meta-analytic model of Calamia, Markon and Tranel (2012) as working in exactly this way: it starts from a baseline practice effect, adds or subtracts weights for the test form, age, retest interval and clinical condition, multiplies the resulting predicted practice effect by 10 (the standard deviation of a T-score) and adds that to the baseline score to predict the follow-up score. These formulas are editorial summaries of the sources, not a published standard equation.

The data table below lists the published quantities used on this page: the Calamia model’s baseline and adjustments, and the retest curve for response times.

Worked example with the arithmetic

Example 1: expected practice effect from published model weights

Duff and Hammers report that the Calamia model’s baseline practice effect is 0.242 SD for a healthy 40-year-old retested on the same form of an auditory attention and working-memory test after one year, with an alternate-form weight of −0.217 and an age weight of −0.004 for every year over 40. Using only those quoted terms (other terms in the full model, such as test domain and interval, are not shown here):

  • Same form, age 40: 0.242 SD. On a T-score metric (SD = 10) that is 0.242 × 10 = 2.42 T-points.
  • Alternate form, age 40: 0.242 − 0.217 = 0.025 SD, or 0.25 T-points.
  • Same form, age 60: 0.242 + (−0.004 × 20) = 0.242 − 0.080 = 0.162 SD, or 1.62 T-points.

The alternate form removes almost all of the modelled gain in this simplified case, and the older participant is expected to gain less than the younger one.

Example 2: separating practice from a treatment effect (illustrative composite)

This example uses invented numbers to show the arithmetic; it is an illustrative composite, not data from a real study. Six participants in a control group and six in a treated group each complete a memory test twice, with the same form both times.

Group Scores at time 1 Scores at time 2 Mean time 1 Mean time 2 Mean change
Control (no treatment) 48, 52, 45, 50, 55, 47 51, 55, 46, 54, 57, 50 49.50 52.17 +2.67
Treated 49, 51, 46, 53, 50, 47 58, 60, 52, 63, 57, 55 49.33 57.50 +8.17
  1. Control change scores (time 2 minus time 1): 3, 3, 1, 4, 2, 3. Sum = 16, mean = 16 / 6 = 2.67. This is the practice effect in this design, because the control group received nothing else.
  2. Treated change scores: 9, 9, 6, 10, 7, 8. Sum = 49, mean = 49 / 6 = 8.17.
  3. Practice-adjusted treatment gain = 8.17 − 2.67 = 5.50 points. Roughly a third of the treated group’s raw gain is practice.
  4. Standardised practice effect in the control group: SD1 = 3.62, so dPE = 2.67 / 3.62 = 0.74.
  5. Paired test on the control changes: SD of the changes = 1.03, standard error = 1.03 / √6 = 0.42, t(5) = 2.67 / 0.42 = 6.32.

How large are practice effects?

The size depends on the test, the person and the interval, so no single number applies. Two meta-analyses read for this page give a sense of scale.

  • Calamia and colleagues’ meta-analysis of 379 studies, as described by Duff and Hammers, gives an average practice effect of 0.242 SD for the baseline case in Example 1.
  • Scharfen and colleagues (36 studies, 49 samples, 212 outcomes, N = 21,810) found that response times in mental speed tests fell by a standardised mean of 0.237 from the first to the second test, 0.367 from the first to the third and 0.499 from the first to the fourth. The largest drop was between the first and second administrations, simple tasks levelled off after the second, and each extra week of interval was associated with about 0.001 SD less gain.

Duff and Hammers list age, education, retest interval, cognitive domain and clinical condition as moderators. Note that the sign differs by outcome: scores on accuracy tests rise with practice, response times fall.

Interpreting a practice effect

What you see Reading Source basis
Large gain from test 1 to test 2, smaller gains afterwards Typical retest curve; most of the practice benefit arrives at the first retest. Scharfen et al. 2018
Gain is similar in treated and control groups The change is consistent with practice (or maturation), not a treatment effect. Control-group logic described by Duff and Hammers 2020
Gain shrinks when an alternate form is used Part of the gain was familiarity with specific items; alternate forms reduce but do not remove it. Calamia model weight as reported by Duff and Hammers 2020
Observed follow-up scores fall below a model’s predicted scores The model may not fit your sample, especially for clinical groups or tests outside the model’s evidence base. Duff and Hammers 2020 (amnestic MCI sample)
Longer retest interval Slightly smaller average gain in response-time tests. Scharfen et al. 2018 (about 0.001 SD per week)

Assumptions and common mistakes

  • Treating a raw change as an effect. Without a comparison group or a practice adjustment, a within-person gain mixes practice, maturation and any intervention. See within-subjects vs between-subjects designs for how the design choice shapes this problem.
  • Applying a published practice effect to the wrong population. Duff and Hammers tested the Calamia model in 93 adults with amnestic mild cognitive impairment (mean age 74.9) retested after about 1.3 years. At group level three of seven observed scores were significantly below prediction (HVLT-R Delayed Recall d = 0.63, BVMT-R Total Recall d = 0.53, BVMT-R Delayed Recall d = 0.40); for HVLT-R Delayed Recall, 40% of predicted scores exceeded observed scores by at least one SD; Lin’s concordance coefficients ranged from 0.15 to 0.71. Their conclusion was caution in MCI, especially on tests that were not in the original meta-analysis.
  • Assuming practice gains keep growing. The response-time evidence points to the biggest gain at the second test and a plateau for simple tasks.
  • Confusing practice with reliability. A test can be reliable across occasions and still show a systematic mean gain. See test-retest reliability.
  • Ignoring the repeated-measures structure when testing the gain. Repeated scores from the same people are dependent; see repeated measures ANOVA assumptions.

How to reduce or control practice effects

Strategy What the sources say
Alternate (parallel) test forms Named by Duff and Hammers as a way to reduce or model practice effects, and by Scharfen et al. as one of the most commonly recommended methods; Scharfen et al. found too few studies to test it for response times.
Dual baseline design Named by Duff and Hammers as a design that can reduce or accurately model practice effects.
Well-matched control group Named by Duff and Hammers as a way to model practice effects.
Standardised regression-based change scores Widely used in neuropsychology according to Duff and Hammers.
Reliable Change Indexes, preferably practice-adjusted Widely used in neuropsychology according to Duff and Hammers; they recommend versions that control for practice.
Avoid reusing the same test with the same person; avoid advance preparation Advice from Scharfen et al.

How to report a practice effect (APA style example)

Report the means and standard deviations at both time points, the test statistic with degrees of freedom, the exact p value and a standardised effect size, and say which form was used. An example using the illustrative control-group data from Example 2:

In the control group, scores rose from time 1 (M = 49.50, SD = 3.62) to time 2 (M = 52.17, SD = 3.97) on the same test form, t(5) = 6.32, p = .001, dz = 2.58. The standardised gain relative to the time 1 SD was 0.74. The treated group gained 5.50 points more than the control group (8.17 vs 2.67).

Here dz is the mean change divided by the SD of the changes (2.67 / 1.03 = 2.58). For the difference between that and the baseline-SD version, see effect size.

How to calculate a practice effect in Excel, R or Python

Each snippet assumes time 1 scores in one column or vector and time 2 scores in another, for the same participants in the same order.

Excel

Mean change:        =AVERAGE(B2:B7)-AVERAGE(A2:A7)
Standardised (SD1): =(AVERAGE(B2:B7)-AVERAGE(A2:A7))/STDEV.S(A2:A7)
Paired t test (p):  =T.TEST(B2:B7,A2:A7,2,1)

R

t1 = c(48, 52, 45, 50, 55, 47)
t2 = c(51, 55, 46, 54, 57, 50)
mean(t2 - t1)                 # practice effect
mean(t2 - t1) / sd(t1)        # standardised by time 1 SD
t.test(t2, t1, paired = TRUE) # paired t test

Python

import numpy as np
from scipy import stats
t1 = np.array([48, 52, 45, 50, 55, 47])
t2 = np.array([51, 55, 46, 54, 57, 50])
pe = (t2 - t1).mean()                 # practice effect
d_pe = pe / t1.std(ddof=1)            # standardised by time 1 SD
t, p = stats.ttest_rel(t2, t1)        # paired t test

Sources

  • Duff, K., and Hammers, D. B. (2020). Practice effects in mild cognitive impairment: A validation of Calamia et al. (2012). The Clinical Neuropsychologist, 36(3), 571-583. https://pmc.ncbi.nlm.nih.gov/articles/PMC7765738 (checked 2026-11-14).
  • Scharfen, J., Blum, D., and Holling, H. (2018). Response time reduction due to retesting in mental speed tests: A meta-analysis. Journal of Intelligence, 6(1), 6. https://doi.org/10.3390/jintelligence6010006 (full text at https://pmc.ncbi.nlm.nih.gov/articles/PMC6480749, checked 2026-11-14).
  • Calamia, M., Markon, K., and Tranel, D. (2012). The Clinical Neuropsychologist, 26(4), 543-570. https://doi.org/10.1080/13854046.2012.680913 (figures cited as reported by Duff and Hammers, 2020).

Data: Published practice-effect and retest-gain quantities in standard-deviation units

Compiled from two openly readable papers: the Calamia, Markon and Tranel (2012) meta-analytic practice-effect model baseline and adjustment weights as reported by Duff and Hammers (2020), and the standardised response-time change by retest number from the Scharfen, Blum and Holling (2018) meta-analysis. Calamia weights are in practice-effect SD units on accuracy-type scores; Scharfen values are standardised mean response-time changes (negative = faster).

Period covered: 2012/2020.

Published practice-effect and retest-gain quantities in standard-deviation units
QuantityValue (standardised units (SD))Condition or termSource
Baseline practice effect0.242Healthy 40-year-old, same test form, one-year retest, auditory attention/working-memory testDuff & Hammers 2020 (citing Calamia et al. 2012)
Alternate-form adjustment-0.217Added to baseline when an alternate form is usedDuff & Hammers 2020 (citing Calamia et al. 2012)
Age adjustment per year over 40-0.004Added for every year of age above 40Duff & Hammers 2020 (citing Calamia et al. 2012)
Mild cognitive impairment adjustment-0.058Clinical-condition weightDuff & Hammers 2020 (citing Calamia et al. 2012)
Alzheimer's disease adjustment-0.517Clinical-condition weightDuff & Hammers 2020 (citing Calamia et al. 2012)
Response-time change, test 1 to test 2-0.237Mental speed tests, standardised mean changeScharfen et al. 2018
Response-time change, test 1 to test 3-0.367Mental speed tests, standardised mean changeScharfen et al. 2018
Response-time change, test 1 to test 4-0.499Mental speed tests, standardised mean changeScharfen et al. 2018

Follow CASRAI

Research-administration guidance, standards updates and independent tool reviews.

Ask CASRAI · free to try

Ask about Practice Effect: Definition, Size and How to Control It

2 free questions a day, no account.

An AI assistant for research: funder rules, compliance, methods and publishing. It links the sources behind its answers and labels answers that come from a web search.

Answers draw on CASRAI's guides and dictionary plus the federal and funder documents we index: Federal Register, Grants.gov, Regulations.gov and UKRI.

Works on this site and inside Claude, Cursor and the AI tools you already use.

Everything CASRAI publishes — this page, the dictionary, the guides and the news — stays open to read, with no account and no card.

Referenced across the research world

University of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logoUniversity of Cambridge logoColumbia University logoCrossref logoUniversity of Edinburgh logoHarvard University logoUniversity of Oxford logoPrinceton University logoStanford School of Medicine logoUniversity College London logoORCID logo
  • University of Cambridge logo
  • Columbia University logo
  • Crossref logo
  • University of Edinburgh logo
  • Harvard University logo
  • University of Oxford logo
  • Princeton University logo
  • Stanford School of Medicine logo
  • University College London logo
  • ORCID logo

View CASRAI adoption →