ARIMA Models for Research Time Series: AR, I, MA, and Box-Jenkins Identification

An ARIMA model is a way of writing a time series’ own past against itself: what a value predicts about the next value, how much the series has to be differenced before that prediction is stable, and how long a random shock keeps echoing forward. Applied researchers reach for it on counts and rates measured repeatedly over time — monthly grant-application volume, weekly clinic visit counts, quarterly enrollment, a citation count tracked year over year — where the goal is either forecasting the next few periods or testing whether an intervention shifted the series’ level or trend.

The problem with most applied treatments is that they stop at the software call. auto.arima() or pmdarima.auto_arima() will return an order and a set of coefficients for almost any series you hand it, with no visible reasoning a reviewer can check. This guide covers the reasoning instead: what each of the three ARIMA components corrects for, how to test whether your series is even a candidate for the model before you fit anything, and how to read the autocorrelation diagnostics that justify the order you choose — the Box-Jenkins identification process the automated search is a shortcut around, not a replacement for.

What ARIMA actually models: three corrections, not three tuning knobs

A plain linear regression assumes each observation is independent of the others. A time series routinely violates that in three distinct ways, and ARIMA has exactly one component for each. Naming the failure each letter corrects for is the difference between choosing an order and guessing one.

AR(p) — corrects for a value that depends on its own recent past

The autoregressive term regresses the current value on its own lagged values: y_t as a function of y_t−1, y_t−2, …, up to lag p. It corrects for momentum — a series where knowing last period’s value genuinely helps predict this period’s, independent of any trend. A research example: a symptom-severity index that persists within a patient from one assessment to the next behaves this way even after any overall trend is removed.

I(d) — corrects for a level or trend that keeps moving

The integration term is not a model component in the usual sense; it is a preprocessing step, applied d times, that subtracts each observation from the one before it (y_t − y_t−1) until the resulting series has a constant mean. It corrects for non-stationarity — a series whose average level itself drifts, which is the normal condition for raw counts of anything that grows, shrinks, or trends over the study period. d = 1 removes a linear trend; d = 2 (rare, and worth being suspicious of) removes a trend in the trend.

MA(q) — corrects for a shock that doesn’t die out immediately

The moving-average term models the current value as a function of past forecast errors — ε_t−1 through ε_t−q — not past values of the series itself. It corrects for a shock whose effect persists for a fixed, short window and then vanishes completely, as opposed to the AR term’s effect, which decays gradually rather than switching off. A policy change that elevates counts for exactly the following two reporting periods and then has zero further effect is an MA(2) signature, not an AR signature.

Putting the three together: ARIMA(p, d, q)

The notation reports all three orders in that fixed sequence. ARIMA(1,1,0) is a series differenced once, with one autoregressive lag and no moving-average term — effectively a random walk with momentum. ARIMA(0,1,1) is differenced once with a shock that echoes for exactly one period and no autoregression. Most series that survive the identification process below land on small orders; p and q above 2 are uncommon outside specialized applications and usually signal that stationarity wasn’t actually achieved, or that seasonality is present and unaddressed (see the note on SARIMA below).

Test for stationarity before you identify anything

Every step that follows — reading a correlogram, fitting coefficients, trusting an information criterion to compare models — assumes the series being analyzed is stationary: constant mean, constant variance, and an autocovariance structure that depends only on the lag between two points, not on where in time you’re standing. Fit the Box-Jenkins identification process to a raw, trending series and the autocorrelation function decays so slowly, at every lag, that it tells you almost nothing about the true AR/MA structure underneath the trend. Stationarity testing is not a formality before the interesting part; it determines d, which is itself one-third of the answer.

Start by looking, then test

Plot the raw series first. A visibly wandering mean or a variance that visibly widens over time (common in count data, where variance often scales with the level — a case for a log or square-root transform before differencing, not after) is usually obvious before any formal test runs. The formal tests exist to confirm what the plot suggests and to catch the cases — a slow, subtle drift, a near-unit-root process — where eyeballing isn’t reliable.

The augmented Dickey-Fuller (ADF) test

The ADF test’s null hypothesis is that the series has a unit root, i.e. is non-stationary; rejecting the null (a small p-value, conventionally < 0.05) is evidence for stationarity. This asymmetry trips people up in exactly the way that matters: failing to reject the ADF null is not evidence of stationarity, it’s an absence of evidence against non-stationarity — a distinction that matters most in the short, noisy series typical of applied research, where the test has limited power to reject a false null in the first place.

The KPSS test — and why you run it alongside ADF, not instead of it

The KPSS test flips the null hypothesis: it assumes the series is stationary, and rejecting the null is evidence against stationarity. Running both tests against the same series is the standard practical fix for the ADF test’s asymmetry problem above — the two disagree in exactly the cases where a single test would mislead you:

• ADF rejects, KPSS fails to reject — both point to stationary. The clean case.
• ADF fails to reject, KPSS rejects — both point to non-stationary. Difference the series.
• Both reject, or neither rejects — the tests disagree or are both inconclusive. This is common in short series and in series with a structural break rather than a smooth trend; report both results and treat the stationarity call as uncertain rather than picking whichever test gave the convenient answer.

Differencing, and the over-differencing trap

Differencing once (y_t − y_t−1) is the standard fix once a series fails the stationarity checks above; re-run both tests on the differenced series before assuming one round was enough. The trap runs the other direction more often than researchers expect: differencing a series that was already stationary, or differencing twice when once was sufficient, introduces artificial negative autocorrelation at lag 1 and inflates the variance of the resulting series rather than reducing it. A differenced series whose ACF shows a sharp negative spike at lag 1 and nothing meaningful afterward is a common signature of exactly this — check whether d was actually necessary before reading anything else off that correlogram.

The Box-Jenkins identification process

Box and Jenkins’ 1970 monograph Time Series Analysis: Forecasting and Control laid out model-building as three iterative stages — identification, estimation, and diagnostic checking — with an explicit loop back to identification if diagnostics fail. The order-selection step (reading ACF and PACF) is the part most guides isolate and present as the whole method; it’s stage one of three, and skipping stages two and three is exactly how a plausible-looking order goes into a paper without ever being checked against its own residuals.

Stage 1 — identification: what the ACF and PACF actually show

Both functions measure correlation between a series and its own lagged values, but they answer different questions:

• ACF (autocorrelation function) at lag k is the raw correlation between y_t and y_t−k, including whatever indirect correlation flows through the lags in between.
• PACF (partial autocorrelation function) at lag k is that same correlation with the effect of lags 1 through k−1 partialed out — the correlation attributable to lag k specifically, and nothing shorter.

That difference is exactly what makes the two functions diagnostic together: an AR process correlates with every earlier lag through the chain of dependence, so its ACF decays gradually across many lags, but once you’ve conditioned on the intervening lags (what PACF does) there’s nothing left to explain beyond lag p — so PACF cuts off sharply. An MA process is, by construction, a function of only the last q shock terms, so its raw correlation with the series vanishes abruptly past lag q (ACF cuts off), while PACF, having to reconstruct a finite-memory process out of an infinite autoregressive representation, decays gradually instead.

Reading the correlogram: the pattern table

The ARMA row is the honest caveat most walkthroughs skip: when both functions tail off, you cannot read p and q off the correlogram by inspection alone. In practice this means starting from a small candidate set (ARMA(1,1) is the default first guess) and letting the diagnostic and information-criterion steps below do the discriminating, rather than staring harder at the plot. Confidence-interval bands (typically ±1.96/√n, shown as dashed lines on most software’s correlogram output) are what “significant spike” means in practice — a bar that crosses the band at a given lag, not a bar that’s merely nonzero.

Stage 2 — estimation

Once a candidate order is identified, coefficient estimation is the mechanical part software handles well: maximum likelihood or conditional sum-of-squares, depending on the package. This stage rarely needs manual judgment; the judgment calls are in stages 1 and 3.

Stage 3 — diagnostic checking: is the residual actually white noise?

A correctly specified ARIMA model should leave nothing but white noise in its residuals — no remaining autocorrelation structure for a higher-order model to pick up. Two checks, both routine:

• Residual ACF/PACF should show no significant spikes at any lag (same ±1.96/√n bands as above, now applied to the residual series rather than the raw one).
• The Ljung-Box test (Ljung and Box, 1978) is the formal, aggregate version of the same check — it tests the joint null hypothesis that residual autocorrelations up to a chosen lag are all zero. A significant Ljung-Box statistic (rejecting the null) means the model has left structure on the table; the order needs revision.

If either check fails, the process returns to stage 1 with a revised candidate order — the loop the three-stage framing exists specifically to formalize, rather than treating identification as a single one-shot guess.

Choosing among candidates that all pass diagnostics

It’s common for more than one (p, d, q) combination to produce clean, white-noise residuals. Compare survivors with an information criterion — AIC or BIC — rather than by eye. Both penalize additional parameters, but BIC penalizes them more heavily as sample size grows, so it tends to favor the more parsimonious model on longer series; report which criterion you used and prefer the simpler model when two candidates are close, since an unnecessarily complex order buys no real forecasting improvement and makes the model harder to defend if a reviewer asks why a particular lag is in the equation.

Where the auto-ARIMA shortcut goes wrong for a methods section

Automated order search (R’s forecast::auto.arima(), Python’s pmdarima.auto_arima()) works by fitting many candidate orders and selecting by AIC, and it is a legitimate, efficient way to screen a wide space of candidates. What it doesn’t do on its own is stand in for the reasoning above: it won’t test stationarity for you in a way you can cite, it won’t show a correlogram a reviewer can inspect, and “the software selected ARIMA(2,1,1)” is not, by itself, a methodological justification. Use it as a starting point or a cross-check against your own identification — and report the stationarity test results, the correlogram-based reasoning (or an explicit note that both ACF and PACF tailed off and the order came from information-criterion search among a stated candidate set), and the diagnostic results regardless of how the candidate order was found.

Beyond plain ARIMA

Seasonality — SARIMA

Monthly or quarterly research series often carry a periodic pattern on top of trend — enrollment that peaks every academic term, clinic visits that dip every December. Plain ARIMA has no way to represent a fixed periodic lag; SARIMA(p,d,q)(P,D,Q)_s adds a second set of orders operating at the seasonal period s (12 for monthly data with an annual cycle, 4 for quarterly). A correlogram with significant spikes recurring at multiples of the seasonal period, after the ordinary ACF/PACF pattern has otherwise settled, is the identification signal that seasonality needs its own term rather than being absorbed into a larger p or q.

External predictors — ARIMAX

When the research question is whether an intervention or covariate shifted the series rather than pure forecasting, ARIMAX adds regression terms for exogenous predictors alongside the ARIMA structure on the errors. The usual regression diagnostics still apply to those predictors — check for multicollinearity among them before interpreting individual coefficients, the same as in any other regression with more than one predictor.

Both extensions are outside this guide’s scope in detail, but knowing the terms exist is what keeps a plain-ARIMA choice deliberate rather than a default reached because it was the first model tried.

What to report in a methods section

• The stationarity test(s) used, their results, and the differencing order (d) applied as a result.
• How the order was identified — correlogram-based reasoning, an automated search, or both — and the candidate set considered.
• The information criterion used to choose among competing candidates, if more than one passed diagnostics.
• Diagnostic check results: the Ljung-Box test statistic and p-value, and a note on residual ACF/PACF.
• Software and package/version (results can differ slightly between implementations’ default estimation methods).
• For forecasts: the forecast horizon and the evaluation metric used to assess accuracy, distinct from the in-sample fit statistics above.

Errors that recur

• Fitting to a raw, non-stationary series and reading a correlogram that’s dominated by trend rather than by the AR/MA structure underneath it.
• Trusting a single stationarity test. ADF and KPSS have opposite nulls for a reason; a result from only one is half the picture.
• Over-differencing a series that was already stationary, then misreading the resulting artificial negative autocorrelation as real MA structure.
• Skipping diagnostic checking. A correlogram-plausible order with residual autocorrelation left over is not a finished model.
• Reading a short, noisy correlogram too literally. Below roughly 50 observations, individual ACF/PACF spikes are unreliable; treat the overall pattern, not any single lag, as the signal, and be more conservative about claiming a clean cutoff.
• Ignoring seasonality in periodic data and trying to force it into a larger, harder-to-justify p or q instead of adding a seasonal term.

Frequently asked questions

What’s the actual difference between ACF and PACF?

ACF at lag k is the raw correlation between a series and its lag-k value, including correlation that flows through shorter lags. PACF at lag k removes the effect of lags 1 through k−1 first, isolating the correlation specific to lag k. AR processes cut off in PACF; MA processes cut off in ACF — that contrast is the whole basis of correlogram-based identification.

How do I know if my series is stationary?

Plot it first, then run both the ADF test (null: non-stationary) and the KPSS test (null: stationary). Treat agreement between the two as a clean result and disagreement as a signal to look more closely, possibly at a structural break rather than a smooth trend, before differencing.

What’s the difference between ARIMA and SARIMA?

SARIMA adds a second (P,D,Q) block operating at a fixed seasonal period on top of the ordinary ARIMA structure, for series with a recurring periodic pattern — monthly, quarterly, or otherwise cyclical research data. Plain ARIMA has no mechanism to represent that periodicity directly.

How many observations do I need to fit an ARIMA model?

There’s no hard cutoff, but correlogram-based identification becomes unreliable below roughly 50 observations, and most applied guidance treats that as a practical floor rather than a comfortable working size — more observations produce a cleaner, more trustworthy correlogram and more stable coefficient estimates, particularly for seasonal models where the seasonal lags themselves eat into the effective sample.

What if the ADF and KPSS tests disagree?

Don’t pick whichever result supports the model you wanted to fit. Disagreement is common with short series and with series that have a structural break rather than a smooth trend; report both results, consider whether a break (rather than differencing) is the better explanation, and be explicit in the methods section that the stationarity call was not clean-cut.

Is auto-ARIMA output good enough to cite on its own in a paper?

The selected order isn’t a justification by itself. Report the stationarity tests, the identification reasoning or candidate set the search covered, and the diagnostic (Ljung-Box, residual ACF/PACF) results regardless of whether the order came from manual correlogram reading, an automated search, or both.

What’s the difference between AIC and BIC for comparing ARIMA candidates?

Both penalize additional parameters against the improvement in fit they buy; BIC’s penalty grows more steeply with sample size, so it tends to favor a more parsimonious model than AIC on longer series. State which one you used rather than reporting “the best-fitting model” without saying by what criterion.

Sources and a note on verification

Box and Jenkins’ 1970 publication date and the three-stage identification/estimation/diagnostic-checking structure were verified this session. The Dickey-Fuller/augmented Dickey-Fuller test, the KPSS test (Kwiatkowski, Phillips, Schmidt, and Shin), and the Ljung-Box test are cited by their standard, widely-taught names and attributions; this guide does not attach specific journal-citation details (volume, page numbers) to those tests beyond the authors’ names, to avoid stating a level of citation precision not independently confirmed in this session. The pattern-table entries (ACF/PACF cutoff versus tailing-off behavior by process type) reflect standard time-series methodology taught consistently across statistics and econometrics references.