Definition · Plain-language
P-value
A p-value is the probability of obtaining a result at least as extreme as the one observed, assuming the null hypothesis is true. A small p-value casts doubt on the null.
The step most authors miss
Doing CRediT right? Don’t stop at the statement.
A CRediT statement credits you inside one paper. The recognition CRediT was built for happens when those roles are tied to you, persistently. Sign in with your ORCID — free — and claim your CRediT contributions on casrai.org, the home of the standard. They become a verified, portable part of your identity, not a line that disappears into one PDF.
Free: claim your contributions, then export a journal-ready CRediT statement, schema.org structured data, JATS XML, CSV or BibTeX — and preview your public profile. A membership publishes that profile publicly and verifies the journals you serve.
What a p-value actually measures
A p-value answers one narrow question: if the null hypothesis were true, how surprising is the result we obtained? Formally it is the probability, computed under the null, of getting a test statistic at least as extreme as the one seen in the data. A very small p-value means such data would rarely arise by chance under the null, which is taken as evidence against it. Crucially, the p-value is a statement about the data given a hypothesis, not about the hypothesis given the data — a distinction that underlies most misuse of the figure.
Comparing p to the significance level
Researchers fix a significance level, α, before collecting data — most commonly 0.05, though 0.01 or 0.001 are used where stronger evidence is required. If the p-value is less than α, the result is declared statistically significant and the null hypothesis is rejected; if it is greater, researchers fail to reject the null. The α threshold is a convention, not a law of nature: 0.05 simply means accepting a one-in-twenty risk of a false positive. Pre-registering the threshold guards against moving the goalposts once results are in.
What a p-value cannot tell you
A p-value does not measure how large, real or important an effect is — a tiny, trivial effect can be highly significant in a large sample, while an important effect can be non-significant in a small one. It is not the probability that the null hypothesis is true, nor the probability the result occurred by chance. The American Statistical Association’s 2016 statement warned against treating p < 0.05 as a hard line for truth, urging researchers to report effect sizes and confidence intervals alongside p-values.
Key facts
At a glance
- Definition: probability of data at least as extreme as observed, assuming H0 is true
- Symbol: p (a probability between 0 and 1)
- Threshold: compared with the significance level α (commonly 0.05)
- Decision: p < α leads to rejecting the null hypothesis
- Does NOT give: the probability the null is true, or the size of an effect
- Caveat: ASA (2016) warns against using p < 0.05 as a sole measure of importance
Common misconceptions
What people often get wrong
Often heard: The p-value is the probability that the null hypothesis is true.
Actually: It is not. A p-value is computed assuming the null is true; it gives the probability of the data under that assumption, not the probability of the hypothesis given the data.
Often heard: A small p-value means the effect is large or important.
Actually: No. Statistical significance and practical importance are separate. A trivial effect can be highly significant in a large sample, which is why effect sizes and confidence intervals should also be reported.
Often heard: A p-value of 0.05 means there is a 5% chance the results occurred by chance.
Actually: That is a loose misreading. The 0.05 is the probability of results at least as extreme as observed if the null were true — a conditional probability about the data, not the chance the result is a fluke.
Going deeper
