Written and maintained by CASRAI Editorial Board
Published
On this page
A practice effect is an improvement in test scores that comes from having taken the same or a similar test before, rather than from a real change in the ability, attitude or condition being measured. It is also called a retest effect. It matters in any study that measures the same people more than once, because a gain between time 1 and time 2 can be partly or wholly practice, and a change score cannot tell the two apart on its own.
What is a practice effect? Definition and synonyms
Duff and Hammers (2020) define practice effects as “improvements in cognitive test scores due to repeated exposure to testing materials.” Scharfen, Blum and Holling (2018) use “retest effects” as their main term, treat “practice effects” as a synonym, and describe them as the gain in test performance that results from repeated administration of the same or an equivalent test.
Why repeated testing raises scores
Scharfen and colleagues, following Lievens and colleagues (2007), separate three routes to a retest gain. The first is a true increase in the latent ability being tested. The second is a reduction in construct-irrelevant factors such as test anxiety or unfamiliarity with the format. The third is the acquisition of test-specific skills or strategies, including automatization of the task. Only the first of these is a change in what you meant to measure; the other two inflate the score without changing the construct.
Practice effect vs maturation vs carryover
These three are often confused because each produces a change between two time points.
- Practice effect: the change is caused by earlier exposure to the measurement itself.
- Maturation: participants change naturally over time (growth, ageing, recovery) whether or not they were tested before. See maturation as a threat to internal validity.
- Carryover: in a crossover design, the effect of an earlier treatment period persists into a later one. See crossover trial design and carryover.
A practice effect can be present with no maturation and no carryover, and vice versa. A control group that is tested on the same schedule is the usual way to separate them from a treatment effect.
Practice effect formula and notation
There is no single standard formula, because a practice effect is a descriptive difference between two administrations. The simplest form is:
PE = M2 − M1
Standardised so it can be compared across tests:
dPE = (M2 − M1) / SD1
- M1: mean score at the first administration.
- M2: mean score at the second administration of the same or an equivalent form.
- SD1: standard deviation of the first-administration scores (some authors use a pooled or normative SD; state which one you used).
When a practice effect is known in advance, an individual’s practice-adjusted change is the observed change minus the expected practice effect:
Adjusted change = (X2 − X1) − PEexpected
Duff and Hammers describe the meta-analytic model of Calamia, Markon and Tranel (2012) as working in exactly this way: it starts from a baseline practice effect, adds or subtracts weights for the test form, age, retest interval and clinical condition, multiplies the resulting predicted practice effect by 10 (the standard deviation of a T-score) and adds that to the baseline score to predict the follow-up score. These formulas are editorial summaries of the sources, not a published standard equation.
The data table below lists the published quantities used on this page: the Calamia model’s baseline and adjustments, and the retest curve for response times.
Worked example with the arithmetic
Example 1: expected practice effect from published model weights
Duff and Hammers report that the Calamia model’s baseline practice effect is 0.242 SD for a healthy 40-year-old retested on the same form of an auditory attention and working-memory test after one year, with an alternate-form weight of −0.217 and an age weight of −0.004 for every year over 40. Using only those quoted terms (other terms in the full model, such as test domain and interval, are not shown here):
- Same form, age 40: 0.242 SD. On a T-score metric (SD = 10) that is 0.242 × 10 = 2.42 T-points.
- Alternate form, age 40: 0.242 − 0.217 = 0.025 SD, or 0.25 T-points.
- Same form, age 60: 0.242 + (−0.004 × 20) = 0.242 − 0.080 = 0.162 SD, or 1.62 T-points.
The alternate form removes almost all of the modelled gain in this simplified case, and the older participant is expected to gain less than the younger one.
Example 2: separating practice from a treatment effect (illustrative composite)
This example uses invented numbers to show the arithmetic; it is an illustrative composite, not data from a real study. Six participants in a control group and six in a treated group each complete a memory test twice, with the same form both times.
| Group | Scores at time 1 | Scores at time 2 | Mean time 1 | Mean time 2 | Mean change |
|---|---|---|---|---|---|
| Control (no treatment) | 48, 52, 45, 50, 55, 47 | 51, 55, 46, 54, 57, 50 | 49.50 | 52.17 | +2.67 |
| Treated | 49, 51, 46, 53, 50, 47 | 58, 60, 52, 63, 57, 55 | 49.33 | 57.50 | +8.17 |
- Control change scores (time 2 minus time 1): 3, 3, 1, 4, 2, 3. Sum = 16, mean = 16 / 6 = 2.67. This is the practice effect in this design, because the control group received nothing else.
- Treated change scores: 9, 9, 6, 10, 7, 8. Sum = 49, mean = 49 / 6 = 8.17.
- Practice-adjusted treatment gain = 8.17 − 2.67 = 5.50 points. Roughly a third of the treated group’s raw gain is practice.
- Standardised practice effect in the control group: SD1 = 3.62, so dPE = 2.67 / 3.62 = 0.74.
- Paired test on the control changes: SD of the changes = 1.03, standard error = 1.03 / √6 = 0.42, t(5) = 2.67 / 0.42 = 6.32.
How large are practice effects?
The size depends on the test, the person and the interval, so no single number applies. Two meta-analyses read for this page give a sense of scale.
- Calamia and colleagues’ meta-analysis of 379 studies, as described by Duff and Hammers, gives an average practice effect of 0.242 SD for the baseline case in Example 1.
- Scharfen and colleagues (36 studies, 49 samples, 212 outcomes, N = 21,810) found that response times in mental speed tests fell by a standardised mean of 0.237 from the first to the second test, 0.367 from the first to the third and 0.499 from the first to the fourth. The largest drop was between the first and second administrations, simple tasks levelled off after the second, and each extra week of interval was associated with about 0.001 SD less gain.
Duff and Hammers list age, education, retest interval, cognitive domain and clinical condition as moderators. Note that the sign differs by outcome: scores on accuracy tests rise with practice, response times fall.
Interpreting a practice effect
| What you see | Reading | Source basis |
|---|---|---|
| Large gain from test 1 to test 2, smaller gains afterwards | Typical retest curve; most of the practice benefit arrives at the first retest. | Scharfen et al. 2018 |
| Gain is similar in treated and control groups | The change is consistent with practice (or maturation), not a treatment effect. | Control-group logic described by Duff and Hammers 2020 |
| Gain shrinks when an alternate form is used | Part of the gain was familiarity with specific items; alternate forms reduce but do not remove it. | Calamia model weight as reported by Duff and Hammers 2020 |
| Observed follow-up scores fall below a model’s predicted scores | The model may not fit your sample, especially for clinical groups or tests outside the model’s evidence base. | Duff and Hammers 2020 (amnestic MCI sample) |
| Longer retest interval | Slightly smaller average gain in response-time tests. | Scharfen et al. 2018 (about 0.001 SD per week) |
Assumptions and common mistakes
- Treating a raw change as an effect. Without a comparison group or a practice adjustment, a within-person gain mixes practice, maturation and any intervention. See within-subjects vs between-subjects designs for how the design choice shapes this problem.
- Applying a published practice effect to the wrong population. Duff and Hammers tested the Calamia model in 93 adults with amnestic mild cognitive impairment (mean age 74.9) retested after about 1.3 years. At group level three of seven observed scores were significantly below prediction (HVLT-R Delayed Recall d = 0.63, BVMT-R Total Recall d = 0.53, BVMT-R Delayed Recall d = 0.40); for HVLT-R Delayed Recall, 40% of predicted scores exceeded observed scores by at least one SD; Lin’s concordance coefficients ranged from 0.15 to 0.71. Their conclusion was caution in MCI, especially on tests that were not in the original meta-analysis.
- Assuming practice gains keep growing. The response-time evidence points to the biggest gain at the second test and a plateau for simple tasks.
- Confusing practice with reliability. A test can be reliable across occasions and still show a systematic mean gain. See test-retest reliability.
- Ignoring the repeated-measures structure when testing the gain. Repeated scores from the same people are dependent; see repeated measures ANOVA assumptions.
How to reduce or control practice effects
| Strategy | What the sources say |
|---|---|
| Alternate (parallel) test forms | Named by Duff and Hammers as a way to reduce or model practice effects, and by Scharfen et al. as one of the most commonly recommended methods; Scharfen et al. found too few studies to test it for response times. |
| Dual baseline design | Named by Duff and Hammers as a design that can reduce or accurately model practice effects. |
| Well-matched control group | Named by Duff and Hammers as a way to model practice effects. |
| Standardised regression-based change scores | Widely used in neuropsychology according to Duff and Hammers. |
| Reliable Change Indexes, preferably practice-adjusted | Widely used in neuropsychology according to Duff and Hammers; they recommend versions that control for practice. |
| Avoid reusing the same test with the same person; avoid advance preparation | Advice from Scharfen et al. |
How to report a practice effect (APA style example)
Report the means and standard deviations at both time points, the test statistic with degrees of freedom, the exact p value and a standardised effect size, and say which form was used. An example using the illustrative control-group data from Example 2:
In the control group, scores rose from time 1 (M = 49.50, SD = 3.62) to time 2 (M = 52.17, SD = 3.97) on the same test form, t(5) = 6.32, p = .001, dz = 2.58. The standardised gain relative to the time 1 SD was 0.74. The treated group gained 5.50 points more than the control group (8.17 vs 2.67).
Here dz is the mean change divided by the SD of the changes (2.67 / 1.03 = 2.58). For the difference between that and the baseline-SD version, see effect size.
How to calculate a practice effect in Excel, R or Python
Each snippet assumes time 1 scores in one column or vector and time 2 scores in another, for the same participants in the same order.
Excel
Mean change: =AVERAGE(B2:B7)-AVERAGE(A2:A7)
Standardised (SD1): =(AVERAGE(B2:B7)-AVERAGE(A2:A7))/STDEV.S(A2:A7)
Paired t test (p): =T.TEST(B2:B7,A2:A7,2,1)
R
t1 = c(48, 52, 45, 50, 55, 47)
t2 = c(51, 55, 46, 54, 57, 50)
mean(t2 - t1) # practice effect
mean(t2 - t1) / sd(t1) # standardised by time 1 SD
t.test(t2, t1, paired = TRUE) # paired t test
Python
import numpy as np
from scipy import stats
t1 = np.array([48, 52, 45, 50, 55, 47])
t2 = np.array([51, 55, 46, 54, 57, 50])
pe = (t2 - t1).mean() # practice effect
d_pe = pe / t1.std(ddof=1) # standardised by time 1 SD
t, p = stats.ttest_rel(t2, t1) # paired t test
Sources
- Duff, K., and Hammers, D. B. (2020). Practice effects in mild cognitive impairment: A validation of Calamia et al. (2012). The Clinical Neuropsychologist, 36(3), 571-583. https://pmc.ncbi.nlm.nih.gov/articles/PMC7765738 (checked 2026-11-14).
- Scharfen, J., Blum, D., and Holling, H. (2018). Response time reduction due to retesting in mental speed tests: A meta-analysis. Journal of Intelligence, 6(1), 6. https://doi.org/10.3390/jintelligence6010006 (full text at https://pmc.ncbi.nlm.nih.gov/articles/PMC6480749, checked 2026-11-14).
- Calamia, M., Markon, K., and Tranel, D. (2012). The Clinical Neuropsychologist, 26(4), 543-570. https://doi.org/10.1080/13854046.2012.680913 (figures cited as reported by Duff and Hammers, 2020).
Data: Published practice-effect and retest-gain quantities in standard-deviation units
Compiled from two openly readable papers: the Calamia, Markon and Tranel (2012) meta-analytic practice-effect model baseline and adjustment weights as reported by Duff and Hammers (2020), and the standardised response-time change by retest number from the Scharfen, Blum and Holling (2018) meta-analysis. Calamia weights are in practice-effect SD units on accuracy-type scores; Scharfen values are standardised mean response-time changes (negative = faster).
Period covered: 2012/2020.
| Quantity | Value (standardised units (SD)) | Condition or term | Source |
|---|---|---|---|
| Baseline practice effect | 0.242 | Healthy 40-year-old, same test form, one-year retest, auditory attention/working-memory test | Duff & Hammers 2020 (citing Calamia et al. 2012) |
| Alternate-form adjustment | -0.217 | Added to baseline when an alternate form is used | Duff & Hammers 2020 (citing Calamia et al. 2012) |
| Age adjustment per year over 40 | -0.004 | Added for every year of age above 40 | Duff & Hammers 2020 (citing Calamia et al. 2012) |
| Mild cognitive impairment adjustment | -0.058 | Clinical-condition weight | Duff & Hammers 2020 (citing Calamia et al. 2012) |
| Alzheimer's disease adjustment | -0.517 | Clinical-condition weight | Duff & Hammers 2020 (citing Calamia et al. 2012) |
| Response-time change, test 1 to test 2 | -0.237 | Mental speed tests, standardised mean change | Scharfen et al. 2018 |
| Response-time change, test 1 to test 3 | -0.367 | Mental speed tests, standardised mean change | Scharfen et al. 2018 |
| Response-time change, test 1 to test 4 | -0.499 | Mental speed tests, standardised mean change | Scharfen et al. 2018 |
Sources
- Duff & Hammers 2020, Practice effects in mild cognitive impairment: A validation of Calamia et al. (2012) (accessed 2026-11-14)
- Scharfen, Blum & Holling 2018, Response time reduction due to retesting in mental speed tests: A meta-analysis (accessed 2026-11-14)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







