Time Series Basics | STAT-101

Data Indexed by Time

A time series is a sequence of observations collected at successive points in time. Unlike a cross-sectional dataset where observations are independent, time series data has a natural ordering — each observation is linked to the ones before and after it. The temperature recorded every hour, daily stock prices, monthly sales figures, and annual population counts are all time series.

Time series analysis is a branch of statistics concerned with understanding temporal structure and using it to make forecasts. It is foundational in finance, economics, signal processing, climatology, and any field where events unfold over time. The key insight is that past values carry information about future values — and extracting that information is both the challenge and the reward.

What Makes Time Series Special

In regular statistics, we assume observations are independent. In a time series, observations are correlated across time — yesterday's value is related to today's. Ignoring this correlation means ignoring valuable signal buried in the temporal structure.

Time series = data + temporal order + autocorrelation

The Four Components

Most real-world time series can be decomposed into four fundamental components. Understanding these components is the first step toward modeling and forecasting.

Trend (T): The long-run direction of the series — is it generally going up, going down, or staying flat? Global average temperature has an upward trend. Horse-drawn carriage sales have a downward trend. GDP in most countries trends upward over decades.

Seasonality (S): Periodic fluctuations that repeat at regular, known intervals. Retail sales spike every December. Ice cream sales peak in summer and dip in winter. Electricity demand is higher on weekday mornings. Seasonality is predictable and tied to the calendar or natural cycles.

Cyclicality (C): Longer-term, irregular fluctuations that are not of fixed period. Business cycles (boom and bust) typically last several years. Unlike seasonality, cycles are not tied to the calendar and their length is not fixed in advance. They are driven by economic forces, social dynamics, and other complex feedback systems.

Noise (ε): The irregular, random variation that remains after removing trend, seasonality, and cycles. Also called the residual or error component. In a well-fitted model, noise should look like white noise — random with no pattern.

Additive and Multiplicative Decomposition

The relationship between these components can be either additive or multiplicative. In the additive model, the components simply add together. In the multiplicative model, they multiply. Choosing the right model depends on whether the seasonal fluctuations grow in proportion to the trend.

Additive Model

Y_t = T_t + S_t + C_t + \varepsilon_t

Use when seasonal swings are roughly constant in size regardless of the trend level

Multiplicative Model

Y_t = T_t \cdot S_t \cdot C_t \cdot \varepsilon_t

Use when seasonal swings grow proportionally with the trend (common in economic data)

If you plot a time series and notice that the seasonal peaks and troughs get larger as the trend increases, use the multiplicative model. If the seasonal variation stays roughly constant in absolute terms, the additive model is appropriate. Alternatively, you can log-transform multiplicative series to convert them to additive ones.

Stationarity

Many time series models require stationarity — a property where the statistical characteristics of the series do not change over time. Formally, a time series is (weakly) stationary if its mean, variance, and autocorrelation structure are constant over time.

Weak Stationarity Conditions

1. Constant mean: E[X_t] = μ for all t

2. Constant variance: Var(X_t) = σ² for all t

3. Autocovariance depends only on lag: Cov(X_t, X_t+k) depends only on k, not on t

A series with a trend is not stationary because its mean changes over time. A series with changing variance (heteroscedasticity) is not stationary either. Making a series stationary is often the first preprocessing step before applying ARIMA and related models. The most common technique is differencing: subtracting consecutive observations to remove the trend.

First Difference

\nabla Y_t = Y_t - Y_{t-1}

Differencing removes the trend by computing the change between successive observations. If one difference is not enough, second differencing can be applied.

Autocorrelation Function (ACF)

The Autocorrelation Function (ACF) measures the correlation of a time series with its own past values at different lags. It answers: how similar is the series to a copy of itself shifted k steps into the past?

Autocorrelation at Lag k

\rho(k) = \frac{\text{Cov}(X_t,\, X_{t+k})}{\text{Var}(X_t)} = \frac{\gamma(k)}{\gamma(0)}

The ACF at lag k is the correlation between observations k time steps apart. ρ(0) = 1 always; values closer to ±1 indicate strong autocorrelation.

Plotting the ACF for all lags up to some maximum produces the ACF plot (also called a correlogram). The shape of this plot is highly informative:

If the ACF decays slowly (many lags have significant correlation), the series is non-stationary and needs differencing. A sinusoidal pattern in the ACF reveals seasonality. An ACF that drops off sharply after lag q suggests a Moving Average (MA) model of order q. An ACF that decays exponentially suggests an Autoregressive (AR) process.

Partial Autocorrelation Function (PACF)

The Partial Autocorrelation Function (PACF) measures the correlation between X_t and X_t-k after removing the linear influence of the intervening lags 1, 2, ..., k-1. In other words, it isolates the direct effect of lag k, controlling for all shorter lags.

Together, ACF and PACF are the primary diagnostic tools for identifying the right ARIMA model order:

ACF & PACF Interpretation Guide

AR(p) process: ACF decays geometrically or oscillates; PACF cuts off sharply after lag p.

MA(q) process: ACF cuts off sharply after lag q; PACF decays geometrically.

ARMA(p,q): Both ACF and PACF decay gradually — neither cuts off cleanly.

Time Series Decomposition in Practice

Classical decomposition proceeds by estimating each component sequentially. First, a moving average is used to estimate the trend. Once the trend is removed, the seasonal component is estimated by averaging the detrended series across periods. What remains is the residual (noise).

Modern decomposition methods like STL (Seasonal and Trend decomposition using Loess) are more robust — they can handle changing seasonal patterns and are resistant to outliers. In Python, statsmodels.tsa.seasonal.seasonal_decompose and STL implement these methods.

Decomposition serves two purposes. First, it provides insight into the structure of the series — is growth accelerating? Is seasonality changing? Second, it can improve forecasting by modeling each component separately and recombining the forecasts.

In the next lesson, we will build on these fundamentals to learn ARIMA models — the workhorse of time series forecasting. You will see exactly how to use ACF and PACF plots to identify the right model order and how to handle non-stationarity through differencing.

Key Takeaways

A time series is data indexed by time; observations are correlated across time, unlike i.i.d. samples.
Every time series can be decomposed into trend, seasonality, cyclicality, and noise — either additively or multiplicatively.
Stationarity (constant mean, variance, and autocovariance structure) is required by most time series models; differencing is the main tool to achieve it.
The ACF measures correlation at each lag; the PACF isolates the direct correlation at each lag after removing shorter-lag effects.
ACF and PACF plots together reveal the order of the underlying AR and MA processes.