Written and maintained by CASRAI Editorial Board
Published · Last updated
On this page
Content validity is the degree to which an instrument’s items adequately sample the full domain of the construct they are meant to measure — judged not by statistics but by subject-matter experts who compare each item against a definition of the domain. A depression screening tool with content validity covers the recognized symptom domains (mood, sleep, appetite, concentration, anhedonia); a statistics final exam with content validity spans the units actually taught rather than over-sampling one topic and ignoring the rest. It is a judgment about coverage, established before data collection, not a correlation computed after it.
This page is the deep dive on content validity specifically: what the underlying domain-sampling logic is, the expert-panel process used to establish it, and the two named methods — the content validity ratio (CVR) and the content validity index (CVI) — used to quantify expert agreement, worked by hand on a small illustrative example. For how content validity relates to the other measurement-validity concepts (construct validity, criterion validity) and to the separate family of design-validity concepts (internal, external, statistical-conclusion validity), see CASRAI’s types of validity in research overview. For the deeper treatment of construct validity as an accumulating evidence framework, see construct validity: definition, evidence types, and threats.
What Content Validity Actually Asks
Content validity is a domain-sampling question: does the set of items an instrument uses represent the full range of the construct’s content, in roughly the right proportions, without leaving out important sub-domains or over-weighting minor ones? It is established through systematic expert judgment, not through statistical analysis of response data — which is what separates it from every other validity type covered on this site. A scale can produce internally consistent, reliable scores (high Cronbach’s alpha) built entirely from items that miss half the construct’s domain; reliability says nothing about whether the content sampled was the right content.
The starting point for any content validity effort is a blueprint, sometimes called a table of specifications: an explicit map of the construct’s sub-domains (and, for achievement/knowledge instruments, the cognitive level expected — recall versus application versus synthesis) against the items intended to measure each one. Without a blueprint, an expert panel has nothing concrete to rate items against beyond their own general impression, which collapses content validity into face validity (see below). Each sub-domain should in turn rest on an operational definition stating how the construct is observed.
Content Validity Examples
These short cases are illustrative, not drawn from a specific published study.
- Education. A grade-level mathematics test has content validity when its items span the skills actually taught in that grade, in roughly the proportions in the curriculum. A test built only from fractions questions fails the criterion even if every item is well written.
- Psychology. A new depression scale that asks only about behaviours (sleep, appetite, activity) and never about mood or interest has a coverage gap: affective symptoms are part of the construct but absent from the items.
- Health surveys. A quality-of-life questionnaire that covers physical and emotional health but omits social functioning is missing a sub-domain an expert panel would be expected to flag during rating.
In each case the fix is the same: write the blueprint, map items to it, and let a panel rate coverage before any respondent data are collected. Where the panel is built iteratively, a Delphi-style consensus process is one way to structure the rounds, and the scale items themselves are often written as a Likert-type format; see the psychometrics overview for the wider measurement context.
Content Validity vs. Face Validity
The two are routinely conflated because both are judgment-based rather than statistical, but they ask different questions and carry different evidentiary weight.
| Dimension | Content validity | Face validity |
|---|---|---|
| Question asked | Do the items systematically sample the full construct domain? | Do the items superficially look like they measure what they claim to? |
| Who judges it | Subject-matter experts, against an explicit domain definition or blueprint | Anyone — experts, respondents, or lay reviewers — on general impression |
| Method | Structured rating (CVR/CVI), a table of specifications, or a Delphi-style consensus process | Informal read-through; “does this look right?” |
| Evidentiary status | Recognized validity evidence in measurement frameworks (e.g., the Standards for Educational and Psychological Testing, AERA/APA/NCME) | Not treated as validity evidence in modern measurement theory — useful for respondent buy-in and item clarity, not for a validity argument |
| Can it be quantified? | Yes — CVR, I-CVI, S-CVI | Not formally; sometimes reported as a simple “looks reasonable” consensus |
In practice, an instrument can have high face validity and weak content validity (items look plausible but leave out entire sub-domains of the construct), or weak face validity and strong content validity (some validated psychopathology and personality scales deliberately obscure their intent with items that don’t look face-valid, specifically to reduce social-desirability bias). Reviewers and funders generally want to see content validity evidence reported; face validity, on its own, is not treated as evidence that an instrument measures what it claims to measure.
How Content Validity Is Established: The Expert-Panel Process
Content validation follows a broadly consistent sequence across fields, whether the instrument is a psychometric scale, a clinical outcome assessment, or a knowledge test:
- Define the construct domain and build a blueprint. Write an explicit definition of the construct and break it into sub-domains or content areas. For an achievement test, this is a table of specifications crossing content topic against cognitive level; for a psychometric scale, it is typically a list of theoretically derived facets or dimensions drawn from the literature.
- Draft an item pool that maps to the blueprint. Generate more items per sub-domain than the final instrument needs, so weak items can be dropped without leaving a sub-domain uncovered.
- Assemble a panel of subject-matter experts. Experts should have relevant content, clinical, or measurement expertise — not be a convenience sample of colleagues. Panels commonly range from three to ten experts; the exact number changes what statistical threshold applies (see below).
- Have each expert rate each item independently. The two dominant rating tasks are the CVR method (each item rated “essential,” “useful but not essential,” or “not necessary”) and the CVI method (each item rated on a 4-point relevance scale). Experts typically also have the option to flag wording problems or suggest missing content.
- Calculate the agreement statistic for each item, and for the scale overall. This is where CVR or CVI is computed — worked examples below.
- Retain, revise, or drop items based on the result, then, where sub-domains lose coverage because an item was dropped, write or select a replacement item and re-rate it before finalizing the instrument.
The Content Validity Ratio (CVR): Lawshe’s Method
The content validity ratio was introduced by C.H. Lawshe in a 1975 paper, “A Quantitative Approach to Content Validity” (Personnel Psychology), originally for job-relevant test items and now used across psychometrics more broadly. Each expert on the panel rates each item as “essential,” “useful but not essential,” or “not necessary” for measuring the construct. The ratio is:
CVR = (ne − N/2) / (N/2)
where ne is the number of panelists who rated the item “essential” and N is the total number of panelists. CVR ranges from −1 (no expert rated the item essential) to +1 (every expert rated it essential); a value of 0 means exactly half the panel considered it essential.
Worked example (illustrative panel data, not from a specific published study). Five experts rate a five-item instrument as essential / useful-but-not-essential / not-necessary. N = 5, so N/2 = 2.5:
| Item | Experts rating “essential” (ne) | CVR = (ne − 2.5) / 2.5 |
|---|---|---|
| Item 1 | 5 of 5 | (5 − 2.5) / 2.5 = 1.00 |
| Item 2 | 4 of 5 | (4 − 2.5) / 2.5 = 0.60 |
| Item 3 | 3 of 5 | (3 − 2.5) / 2.5 = 0.20 |
| Item 4 | 2 of 5 | (2 − 2.5) / 2.5 = −0.20 |
| Item 5 | 1 of 5 | (1 − 2.5) / 2.5 = −0.60 |
Reading this table: Item 1 has unanimous “essential” agreement and is clearly retained; Items 4 and 5 have a majority saying “not necessary” and are dropped or substantially rewritten; Items 2 and 3 are the judgment calls a panel chair has to resolve. Lawshe’s 1975 paper published a table of minimum CVR values by panel size, and because CVR is bounded by how few experts can disagree before the ratio turns negative, small panels require something close to unanimous “essential” ratings to clear any reasonable minimum — a single dissenting expert on a 5-person panel already pulls an item’s CVR down to 0.60. Note for anyone applying this in practice: subsequent methodological work (Wilson, Pan & Schumsky, 2012; Ayre & Scally, 2014) identified and revised errors in Lawshe’s original critical-value table, and the two corrections don’t fully agree with each other — check current guidance for the minimum value appropriate to your panel size rather than relying on a single fixed cutoff.
The Content Validity Index (CVI): I-CVI and S-CVI
The content validity index uses a different rating task and reporting convention, most associated with Denise Polit and Cheryl Beck’s methodological work (building on C.J. Davis, 1992). Each expert rates each item on a 4-point relevance scale (typically 1 = not relevant, 2 = somewhat relevant, 3 = quite relevant, 4 = highly relevant). An item’s item-level CVI (I-CVI) is the proportion of experts who rated it 3 or 4:
I-CVI = (number of experts rating the item 3 or 4) / (total number of experts)
The commonly cited acceptability threshold (Lynn, 1986; Polit & Beck, 2006) is I-CVI ≥ 0.78 for panels of six or more experts, or ≥ 0.83 for smaller panels of three to five. The scale-level CVI (S-CVI/Ave) is the average of all item I-CVIs, with ≥ 0.90 generally cited as the threshold for excellent scale-level content validity (Polit, Beck & Owen, 2007).
Worked example (illustrative panel data, not from a specific published study). Six experts rate a six-item instrument on the 4-point scale; the table shows how many of the six rated each item 3 or 4:
| Item | Experts rating 3 or 4 | I-CVI = count / 6 | Meets 0.78 threshold? |
|---|---|---|---|
| Item 1 | 6 | 1.00 | Yes |
| Item 2 | 5 | 0.83 | Yes |
| Item 3 | 6 | 1.00 | Yes |
| Item 4 | 3 | 0.50 | No — drop or revise |
| Item 5 | 6 | 1.00 | Yes |
| Item 6 | 4 | 0.67 | No — revise and re-rate |
Averaging all six I-CVIs gives S-CVI/Ave = (1.00 + 0.83 + 1.00 + 0.50 + 1.00 + 0.67) / 6 = 0.833, below the 0.90 target. Dropping Item 4 (the clearest failure) and recomputing across the remaining five items raises it to (1.00 + 0.83 + 1.00 + 1.00 + 0.67) / 5 = 0.90 — which is exactly the mechanism content validation is supposed to produce: removing or replacing the weakest items measurably improves the scale-level content validity estimate, and Item 6 would still need revision and re-rating before finalizing the instrument even though the aggregate now clears the target.
CVI in Research: Meaning, I-CVI, S-CVI/Ave and S-CVI/UA
In research reports, CVI means content validity index: a number between 0 and 1 summarising how many expert raters judged items relevant. It is reported at two levels. The I-CVI (also written i-CVI) is the proportion of experts rating a single item 3 or 4 on the 4-point relevance scale. The S-CVI summarises the whole scale, and it can be computed in two different ways, which is why the method must be named when you report it.
| Statistic | How it is computed | Notes |
|---|---|---|
| I-CVI | Experts rating the item 3 or 4, divided by all experts | Item-level; thresholds of 0.78 (six or more experts) and 0.83 (three to five) are described above |
| S-CVI/Ave | Average of all I-CVIs | The more lenient scale-level figure; 0.90 is the commonly cited target |
| S-CVI/UA | Proportion of items rated 3 or 4 by every expert (universal agreement) | The stricter scale-level figure; a single dissenting expert removes the item from the count |
The two scale-level methods can disagree sharply. In the hypothetical example in Polit and Beck’s critique, three experts rate ten items and seven items win unanimous approval: S-CVI/UA is 0.70 while S-CVI/Ave is 0.90 (Polit & Beck, 2006, Research in Nursing & Health). The authors’ advice is to state which method was used so readers can interpret the figure.
CVR vs CVI: What Is the Difference?
CVR and CVI both quantify expert agreement, and they are often confused (searches such as “CVI vs CVR” and “CVI and CVR difference” are common). The differences are in the rating task and the unit of agreement.
| Dimension | CVR (Lawshe) | CVI (Davis; Lynn; Polit & Beck) |
|---|---|---|
| Rating task | Three categories: essential, useful but not essential, not necessary | 4-point relevance scale (1 = not relevant to 4 = highly relevant) |
| Agreement counted | Experts choosing “essential” | Experts choosing 3 or 4 |
| Range | −1 to +1 | 0 to 1 |
| Typical use | Selecting or dropping items, especially for job-related and knowledge tests | Reporting item and scale-level content validity, common in nursing and health scales |
| Scale-level summary | Usually the mean CVR of retained items | S-CVI/Ave or S-CVI/UA |
Use CVR when the question is which items are essential; use CVI when you want a relevance rating and item- and scale-level indices to report. Whichever you choose, pair it with a cited threshold and a reliability check such as Cronbach’s alpha; for questionnaire drafting see the questionnaire design guide.
How Many Experts Do You Need?
There is no single fixed rule, but the methodological literature converges on a practical range. Panels smaller than three experts are generally considered too small to produce a meaningful agreement statistic; panels larger than about ten to fifteen experts rarely add proportional value and become harder to recruit and coordinate. A panel of five to ten content experts, plus at least one measurement/methods specialist where the instrument will be used for research (as distinct from a purely clinical or educational panel), is a common practical target. Panel size also directly changes which statistical threshold applies — a 5-person panel needs a higher CVR to clear Lawshe’s minimum than a 10-person panel does, and CVI’s own recommended I-CVI threshold shifts from 0.83 to 0.78 once the panel reaches six experts — so panel size should be decided before the rating task is designed, not adjusted afterward to make the numbers work.
Limitations of Content Validity Evidence
Content validity evidence is necessary but not sufficient on its own. It does not establish that the instrument produces consistent scores (reliability), that it correlates with related or unrelated constructs the way theory predicts (construct validity, in the convergent/discriminant sense), or that it predicts an external outcome (criterion validity). An expert panel can also share the same blind spot — if every expert on the panel comes from the same theoretical tradition or the same clinical setting, their agreement demonstrates consensus within that tradition, not necessarily complete domain coverage. This is why methods sections increasingly expect content validity to be reported alongside, not instead of, reliability and other validity evidence — see CASRAI’s Cronbach’s alpha guide for the companion reliability statistic most often reported next to it, and the construct validity guide for the evidence types that typically follow a content validation step.
Chance Agreement and the Modified Kappa
The item-level content validity index (I-CVI) is a simple proportion: experts rating an item 3 or 4 on a 4-point relevance scale, divided by the number of experts. Polit, Beck and Owen (2007, Research in Nursing & Health, 30(4), 459–467) list its advantages (easy to compute, easy to understand, focused on items) and name its main weakness: it does not adjust for the agreement that would occur by chance. They propose a modified kappa statistic that adjusts each I-CVI for chance agreement, and recommend that, with three or more experts, items with an I-CVI of 0.78 or higher be treated as good. Because the adjustment matters most for small panels, report the modified kappa alongside the I-CVI when your panel is small.
A worked check with a six-expert panel: five experts rating an item 3 or 4 gives I-CVI = 5/6 = 0.83, which clears 0.78; four of six gives 0.67, which does not. With nine experts, seven of nine gives 0.78 exactly, the lowest passing count. The data table below lists the smallest passing count for panels from 5 to 40 experts.
Which Critical Values for the CVR?
Lawshe (1975, Personnel Psychology, 28(4), 563–575) published a table of minimum CVR values by panel size so that agreement beyond what chance would produce can be identified. The commonly reproduced values fall as panels grow (0.99 for 5 to 7 panelists, 0.62 for 10, 0.42 for 20, 0.29 for 40), but the table is not smooth: the value shown for 8 panelists (0.75) is lower than the value for 9 (0.78). With 8 experts, 7 rating an item essential gives CVR = (7 − 4)/4 = 0.75, which the table accepts, whereas 7 of 8 is a smaller share than 8 of 9. Wilson, Pan and Schumsky (2012, Measurement and Evaluation in Counseling and Development, 45(3), 197–210) published recalculated critical values, and Ayre and Scally (2014, Measurement and Evaluation in Counseling and Development, 47(1), 79–86) argued in “Critical Values for Lawshe’s Content Validity Ratio” for exact values instead. When you apply a threshold, name the source of your critical values. The data table below sets the commonly reproduced Lawshe values beside the agreement counts they imply and the I-CVI of 0.78 rule.
Content Validity for Patient-Reported Measures: The COSMIN View
In health measurement, the COSMIN initiative treats content validity as the most important measurement property of a patient-reported outcome measure. Terwee et al. (2018, Quality of Life Research, 27(5), 1159–1170) report a Delphi study with 159 experts from 21 countries that reached consensus (67% agreement) on ten criteria for good content validity. The criteria cover three questions: whether items are relevant to the construct, population and context of use (including response options and recall period); whether the instrument is comprehensive, with no key concepts missing; and whether items are comprehensible to the people who answer them. The third question cannot be answered by an expert panel alone, which is why this framework expects input from the target population, for example through cognitive interviews, in addition to professional judgment.
Reporting Content Validity in a Methods Section
A methods section claiming content validity should generally report: how the construct domain and blueprint were defined and from what source (literature review, existing framework, clinical guideline); the number and qualifications of the expert panel; the exact rating task used (CVR’s three-category “essential” judgment, or CVI’s 4-point relevance scale); the resulting item-level and scale-level statistics against a named, cited threshold; and what was done with items that failed to clear it (dropped, revised and re-rated, or retained with justification). Simply stating that “content validity was established by expert review” without any of the above is a common weakness reviewers flag, because it reports that a process happened without reporting what it found.
Two further reporting points: state which scale-level method you used, because Polit and Beck (2006, Research in Nursing & Health, 29(5), 489–497) distinguish S-CVI by universal agreement from S-CVI by averaging and the two give different values for the same ratings; and state the panel size and the rating scale, since both determine which threshold applies.
Frequently Asked Questions
What is content validity in research?
Content validity is the extent to which an instrument’s items adequately and representatively sample the full domain of the construct being measured, as judged by subject-matter experts against an explicit definition of that domain — not by statistical analysis of response data.
What is the difference between content validity and face validity?
Content validity is a structured judgment by subject-matter experts against an explicit domain definition or blueprint, and is recognized as formal validity evidence. Face validity is an informal impression of whether items “look like” they measure the construct, made by anyone, and is not treated as validity evidence in modern measurement frameworks.
What is a good content validity index (CVI) score?
Polit, Beck and Owen (2007) recommend an item-level I-CVI of 0.78 or higher when the panel has three or more experts, and a scale-level S-CVI/Ave of 0.90 or higher is the commonly cited standard for excellent overall content validity (Polit & Beck, 2006). For six experts, 5 of 6 agreeing gives 0.83, which passes.
What is the content validity ratio (CVR)?
CVR is Lawshe’s (1975) statistic quantifying expert-panel agreement that an item is “essential”: CVR = (ne − N/2) / (N/2), where ne is the number of experts rating the item essential and N is the panel size. It ranges from −1 to +1.
How many experts should be on a content validity panel?
There’s no single fixed number, but five to ten subject-matter experts is a common practical range; fewer than three is generally considered too small to produce a meaningful statistic, and panel size determines which CVR/CVI threshold applies.
Is content validity the same as construct validity?
No. Content validity concerns whether items adequately sample the construct’s domain, established through expert judgment before data collection. Construct validity is broader and evidence-based, established through patterns of correlation (convergent, discriminant, and related evidence) after data has been collected — see CASRAI’s construct validity guide for the full evidence framework. Some measurement frameworks, including the Standards for Educational and Psychological Testing, treat content and criterion evidence as contributing to an overall construct validity argument rather than as fully separate categories.
What is an acceptable I-CVI for 6 experts?
Five of six experts rating the item 3 or 4 gives I-CVI = 0.83, which meets the 0.78 benchmark of Polit, Beck and Owen (2007); four of six gives 0.67, which does not.
Why is the Lawshe table inconsistent at 8 panelists?
As commonly reproduced, the table lists 0.75 for 8 panelists and 0.78 for 9, so the critical value rises where it should keep falling. Wilson, Pan and Schumsky (2012) recalculated the values and Ayre and Scally (2014) argued for exact values; cite whichever source you use.
Is the CVI adjusted for chance agreement?
No. Polit, Beck and Owen (2007) identify this as its main weakness and propose a modified kappa that adjusts I-CVI for chance agreement.
What does COSMIN mean by content validity?
COSMIN judges the relevance, comprehensiveness and comprehensibility of a patient-reported measure, using ten criteria agreed in a Delphi study (Terwee et al., 2018), with input from the target population as well as experts.
Sources
- Polit DF, Beck CT, Owen SV (2007). Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Research in Nursing & Health 30(4):459–467. doi:10.1002/nur.20199
- Polit DF, Beck CT (2006). The content validity index: are you sure you know what’s being reported? Research in Nursing & Health 29(5):489–497. doi:10.1002/nur.20147
- Lawshe CH (1975). A quantitative approach to content validity. Personnel Psychology 28(4):563–575. doi:10.1111/j.1744-6570.1975.tb01393.x
- Wilson FR, Pan W, Schumsky DA (2012). Recalculation of the critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development 45(3):197–210. doi:10.1177/0748175612440286
- Ayre C, Scally AJ (2014). Critical values for Lawshe’s content validity ratio. Measurement and Evaluation in Counseling and Development 47(1):79–86. doi:10.1177/0748175613513808
- Terwee CB et al. (2018). COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study. Quality of Life Research 27(5):1159–1170. doi:10.1007/s11136-018-1829-0
- Content validity (Lawshe CVR table as reproduced), Wikipedia, en.wikipedia.org/wiki/Content_validity
Sources (accessed 2026-11-14).
For the broader map of how content validity relates to design-validity concepts like internal and external validity, see types of validity in research and the internal vs. external validity comparison. For the reliability side of instrument evaluation, see Cronbach’s alpha and test-retest vs. inter-rater reliability.
Data: Panel size, Lawshe minimum CVR and the agreement count each criterion implies
For panels of 5 to 40 content experts, compares the minimum content validity ratio (CVR) published in Lawshe's critical-value table as reproduced in reference summaries with the smallest number of experts that must rate an item essential to reach it, and with the smallest number rating an item relevant to reach an item-level CVI (I-CVI) of 0.78, the benchmark from Polit, Beck and Owen (2007). Counts and ratios are computed by the CVR formula (ne - N/2)/(N/2) and I-CVI = agreeing/N.
Period covered: 1975/2007.
| Panel size | Lawshe minimum CVR | Experts rating essential needed (Lawshe) (experts) | CVR at that count | Experts rating relevant needed for I-CVI >= 0.78 (experts) | I-CVI at that count |
|---|---|---|---|---|---|
| 5 panelists | 0.99 | 5 | 1 | 4 | 0.8 |
| 6 panelists | 0.99 | 6 | 1 | 5 | 0.83 |
| 7 panelists | 0.99 | 7 | 1 | 6 | 0.86 |
| 8 panelists | 0.75 | 7 | 0.75 | 7 | 0.88 |
| 9 panelists | 0.78 | 8 | 0.78 | 8 | 0.89 |
| 10 panelists | 0.62 | 9 | 0.8 | 8 | 0.8 |
| 11 panelists | 0.59 | 9 | 0.64 | 9 | 0.82 |
| 12 panelists | 0.56 | 10 | 0.67 | 10 | 0.83 |
| 20 panelists | 0.42 | 15 | 0.5 | 16 | 0.8 |
| 30 panelists | 0.33 | 20 | 0.33 | 24 | 0.8 |
| 40 panelists | 0.29 | 26 | 0.3 | 32 | 0.8 |
Sources
- Content validity (Lawshe CVR formula and critical-value table) (accessed 2026-11-14)
- Polit, Beck & Owen (2007) Is the CVI an acceptable indicator of content validity? (abstract, Europe PMC) (accessed 2026-11-14)
Compiled by CASRAI from the sources above. Free to reuse under CC BY 4.0 with attribution.







