ARIMA Models
The workhorse of time series forecasting. Combining autoregression, differencing, and moving averages into one powerful framework.
Three Components
- AR(p) — predict from p past values
- I(d) — d rounds of differencing for stationarity
- MA(q) — model q past forecast errors
Autoregressive AR(p)
Predict the current value as a weighted sum of p past values plus white noise. The series has memory.
Moving Average MA(q)
Model shocks: the current value depends on q past forecast errors. Impact dies out after exactly q steps.
ACF & PACF Signatures
- AR(p): ACF decays, PACF cuts off after lag p
- MA(q): ACF cuts off after lag q, PACF decays
- ARMA: both ACF and PACF decay gradually
Differencing
Non-stationary series must be differenced d times before fitting. d=1 removes a linear trend; d=2 removes quadratic.
ARIMA (p, d, q)
Apply ARMA(p,q) to the d-times-differenced series. In practice d is 0, 1, or 2.
Identification
Use ADF to determine d, then inspect ACF and PACF of the differenced series to select p and q.
- ADF test → choose d
- ACF / PACF plots → choose p, q
- Plot the series itself for obvious patterns
Estimation & Selection
Fit by maximum likelihood. Compare models with AIC or BIC — both penalize complexity. Lower is better.
Diagnostic Checking
Residuals should look like white noise: no autocorrelation, roughly normal, constant variance. Ljung–Box test checks for remaining structure.
SARIMA
Adds a seasonal layer (P, D, Q)m at period m. Seasonal spikes in ACF at lags m, 2m, 3m signal the need for seasonal terms.
What you learned
ARIMA unifies autoregression, differencing, and moving averages. The Box–Jenkins cycle — identify, estimate, check — gives you a principled workflow for any time series.