Written and maintained by CASRAI Editorial Board
Last updated
A categorical distribution describes a single random outcome that lands in exactly one of K categories, where each category has a fixed probability and the probabilities add up to 1. It is the distribution behind one roll of a die, one survey answer from a list of options, or one word drawn from a vocabulary. This guide gives the formula, a worked example with the arithmetic shown, the exact bridge to NumPy, SciPy, R and Stan, and the mistakes that cause errors in practice.
Categorical distribution definition
A categorical distribution is a discrete probability distribution over K unordered categories for a single trial. It is the generalisation of the Bernoulli distribution from two outcomes to K outcomes. Stan’s Functions Reference defines it as follows: for a parameter vector θ that forms an N-simplex (non-negative entries summing to one), Categorical(y | θ) = θy for y in {1, …, N}.
The same object is often called a multinoulli distribution or a generalised Bernoulli distribution. When you count outcomes over n independent trials instead of one, you get the multinomial distribution; the categorical distribution is that case with n = 1.
Categorical distribution formula (PMF)
For categories labelled 1 to K with probabilities p1, …, pK:
P(Y = i) = pi, with pi ≥ 0 for every i and p1 + p2 + … + pK = 1.
- Y is the outcome of one trial, a category label from 1 to K.
- i is the particular category you are asking about.
- pi is the probability of category i, which is also the PMF value at i.
- K is the number of categories.
If you record the outcome as a one-hot vector x (a 1 in the position of the observed category, 0 elsewhere), the same PMF is written P(x) = p1x1 × … × pKxK. Only one exponent is 1 and the rest are 0, so the product collapses to pi. This is the multinomial PMF, n!/(x1! … xK!) × p1x1 … pKxK, as given in the SciPy documentation, with n = 1: the factorial term is 1!/(x1! … xK!) = 1, because every xj! is 1! or 0!.
Categorical vs Bernoulli vs binomial vs multinomial
| Distribution | Categories (K) | Trials (n) | What it models |
|---|---|---|---|
| Bernoulli | 2 | 1 | One yes/no outcome |
| Binomial | 2 | n | Number of successes in n trials |
| Categorical | K | 1 | Which category one trial lands in |
| Multinomial | K | n | Counts per category over n trials |
The SciPy documentation notes that for K = 2 the multinomial is identical to the corresponding binomial, and the NumPy documentation describes the multinomial as the multivariate generalisation of the binomial. Set n = 1 in either direction and the table collapses to the categorical and Bernoulli rows.
Worked example: estimating the probabilities from 60 die rolls
Suppose a six-sided die is rolled 60 times and the counts for faces 1 to 6 are 8, 11, 9, 12, 7 and 13. (The data are illustrative.) Each roll is one draw from a categorical distribution with K = 6, so the unknown probabilities p1 … p6 can be estimated by the observed proportions p̂i = ni / N.
Step 1: total. N = 8 + 11 + 9 + 12 + 7 + 13 = 60.
Step 2: proportions.
| Face i | Count ni | p̂i = ni / 60 | ni × ln p̂i |
|---|---|---|---|
| 1 | 8 | 0.1333 | −16.119 |
| 2 | 11 | 0.1833 | −18.661 |
| 3 | 9 | 0.1500 | −17.074 |
| 4 | 12 | 0.2000 | −19.313 |
| 5 | 7 | 0.1167 | −15.039 |
| 6 | 13 | 0.2167 | −19.882 |
| Total | 60 | 1.0000 | −106.088 |
Step 3: probability of a specific sequence. Rolls are independent, so the probability of seeing the sequence 4, 2, 4 under these estimates is p̂4 × p̂2 × p̂4 = 0.2 × 0.1833 × 0.2 = 0.00733. In general, the likelihood of all N observations is the product of pini, which is why the log-likelihood is the sum of ni ln pi in the last column: −106.088.
Step 4: compare with a fair die. If every face had probability 1/6, the log-likelihood of the same data is 60 × ln(1/6) = 60 × (−1.7918) = −107.506. The fitted proportions give a higher value (−106.088), as they must, because the observed proportions are the maximum-likelihood estimates. The gap is 1.417 log-units. Whether it is larger than chance would produce is a question for a formal test; for that, see the goodness-of-fit problems in chi-squared examples.
Interpreting the result
| What you see | How to read it |
|---|---|
| p̂i for one category | The estimated chance that a single new trial falls in category i |
| All p̂i add to 1 | The categories are exhaustive and exclusive; if they do not add to 1, a category is missing or double-counted |
| A category with p̂i = 0 | Not observed in the sample; it does not by itself prove the true probability is 0 |
| Log-likelihood closer to 0 | The probabilities explain the data better; only comparable between models fitted to the same data |
Assumptions and common mistakes
- Probabilities must be non-negative and sum to 1. NumPy’s multinomial sampler raises a ValueError for pvals such as [1.0, 2.0]; the valid version is [1/3, 2/3]. R’s rmultinom normalises a non-negative prob vector internally, so check which behaviour your software has.
- Categories are unordered. The labels 1 to K are names, not measurements. Do not average them. If the categories have a natural order, an ordinal model is usually the better description.
- One trial, one category. If a respondent can tick several boxes, the outcome is not categorical.
- Trials are independent and share the same probabilities. Repeated measures on the same person or a probability that drifts over time break this.
- Do not confuse it with multinomial logistic regression. That is a regression model whose outcome has a categorical distribution; the categorical distribution is only its error structure. Stan’s categorical_logit applies a softmax to a vector of real-valued scores to produce the probabilities.
Categorical distribution of a variable (frequency table)
In everyday data work, people also say “the categorical distribution of a variable” to mean its frequency table: the count or percentage in each category, as in the table above. That is the empirical distribution of your sample; the probability model above is what generates it. For examples of categorical variables see categorical variable examples, and for percentages within groups see the conditional relative frequency table guide.
How to report it
State the sample size, the categories and the proportions, and say that they are estimates. An example in the style of a results sentence: “Across 60 rolls, the faces occurred with estimated proportions of .13, .18, .15, .20, .12 and .22 (n = 8, 11, 9, 12, 7, 13).” If you tested fit against a fair die, add the test statistic, degrees of freedom and p-value from that test; a chi-square test is explained in the chi-square test dictionary entry.
Categorical distribution in Excel, R, Python and Stan
Excel
With counts in B2:B7: put the proportion in C2 as =B2/SUM($B$2:$B$7) and fill down. The log-likelihood is =SUMPRODUCT(B2:B7,LN(C2:C7)), which returns −106.088 for the example.
Python (NumPy and SciPy)
import numpy as np
from scipy.stats import multinomial
counts = np.array([8, 11, 9, 12, 7, 13])
p = counts / counts.sum()
rng = np.random.default_rng(1)
draws = rng.multinomial(1, p, size=5) # one-hot rows, n = 1
labels = draws.argmax(axis=-1) + 1 # back to faces 1-6
print(multinomial(1, p).pmf([0, 0, 0, 1, 0, 0])) # 0.2 = p[3]
The NumPy documentation uses n = 1 for categorical draws and recovers the categories with argmax along the last axis. The SciPy pmf with n = 1 returns the single probability pi.
R
x <- c(8, 11, 9, 12, 7, 13)
p <- x / sum(x)
rmultinom(1, size = 1, prob = p) # one one-hot draw
dmultinom(c(0, 0, 0, 1, 0, 0), prob = p) # 0.2
Stan
In Stan, y ~ categorical(theta); states the model, where theta is a simplex; categorical_lpmf gives the log probability and categorical_rng(theta) draws a value. categorical_logit(beta) is the same as categorical(softmax(beta)).
FAQ
Is the categorical distribution the same as multinomial with n = 1?
Yes. A multinomial with one trial returns a one-hot vector, and its PMF reduces to pi for the category that occurred. The difference is in what you record: a single label, or counts across many trials.
How many parameters does a categorical distribution have?
K probabilities, but because they must sum to 1 only K − 1 are free. The die example has 6 probabilities and 5 free parameters.
How do you estimate the probabilities?
Divide each category count by the total number of trials, as in the worked example (p̂i = ni / N). These proportions maximise the likelihood.
Is the categorical distribution the same as a Bernoulli distribution?
Only when K = 2. A Bernoulli has two outcomes; a categorical has K.
Sources
- Stan Development Team, Stan Functions Reference, Bounded Discrete Distributions (categorical and categorical logit): https://mc-stan.org/docs/functions-reference/bounded_discrete_distributions.html (checked 2026-10-29).
- NumPy documentation, numpy.random.Generator.multinomial: https://numpy.org/doc/stable/reference/random/generated/numpy.random.Generator.multinomial.html (checked 2026-10-29).
- SciPy documentation, scipy.stats.multinomial: https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.multinomial.html (checked 2026-10-29).
- R documentation, stats::Multinomial (rmultinom, dmultinom): https://stat.ethz.ch/R-manual/R-devel/library/stats/html/Multinom.html (checked 2026-10-29).
