Module 2 — ARMA, ARIMA, and Model Identification

Core question

Can a small number of lagged values and past shocks explain the serial dependence that remains after trend and seasonality are handled?

Learning outcomes

You will be able to:

• write AR, MA, ARMA, and ARIMA models with the lag operator;
• check AR causality/stationarity and MA invertibility from polynomial roots;
• construct an AR companion matrix and solve its Yule–Walker equations;
• derive benchmark ACF and PACF patterns;
• explain why invertibility is an identification restriction;
• detect under- and over-differencing;
• turn plots into candidate models without treating them as proofs.

1. One notation, three model classes

Let $BX_t=X_{t-1}$. Define

Then

The intercept in software is not always the unconditional mean. For

the stationary mean is $\mu=c/(1-\phi)$.

2. AR means decaying feedback

For AR(1),

It is causal and stationary when $|\phi|<1$, with

For AR($p$), all roots of $\phi(z)=0$ must lie outside the unit circle. Equivalently, the companion-matrix eigenvalues lie inside it.

Worked AR(2) root check

Consider

The polynomial is $1-0.75z+0.125z^2=(1-0.5z)(1-0.25z)$. Its roots are 2 and 4, both outside the unit circle, so the process is stationary. Two positive decay rates create a smoother ACF than a single cutoff rule could describe.

3. Companion and Yule–Walker forms

For AR($p$), define

where the first row of $F$ is $(\phi_1,\ldots,\phi_p)$, its first subdiagonal contains ones, and $G=(1,0,\ldots,0)^\top$.

Three stability statements are equivalent:

The same coefficients solve the moment equations

R: verify all three representations

The polynomial roots are reciprocals of the companion eigenvalues. The Toeplitz solve returns the original coefficients because it uses population autocorrelations.

4. MA means finite shock memory

For MA(1),

With $\theta=0.5$, $\rho(1)=0.4$. A shock affects two observations, so the theoretical ACF cuts off after lag 1.

Why invertibility is needed

$\theta=2$ and $\theta=1/2$ generate the same autocorrelation because

Invertibility selects the representation whose MA roots lie outside the unit circle—here $|\theta|<1$ under this sign convention—so innovations are recoverable from current and past observations and parameters are identified.

5. ACF and PACF are signatures, not verdicts

“Cuts off” is a population result. A sample ACF has noise of roughly order $1/\sqrt n$ under simple white-noise conditions, and inspecting many lags creates multiple-comparison ambiguity. Use ACF/PACF to propose a small candidate set, then estimate, diagnose, and backtest it.

6. Differencing changes the model

First and seasonal differences are

Over-differencing stationary white noise gives $\Delta X_t=\varepsilon_t-\varepsilon_{t-1}$, an MA(1) boundary case with $\rho(1)=-0.5$. A strong negative lag-1 ACF after transformation is a warning.

7. See the signatures, then disturb them

Edit phi, theta, and n. Small samples show why clean textbook cutoffs often look ambiguous.

8. Identification workflow

1. Define target, frequency, origin, and horizon.
2. Plot levels; mark trend, seasonality, breaks, and missing values.
3. Choose transformations for stated reasons.
4. Inspect ACF/PACF and propose at most a few parsimonious orders.
5. Check estimated roots and near-cancellation between AR and MA factors.
6. Audit residual dependence and parameter stability.
7. Compare against naive and seasonal-naive forecasts on rolling origins.
8. Prefer the smallest model whose remaining errors are not predictably useful.

Practice

1. For $X_t=0.6X_{t-1}+\varepsilon_t$, calculate $\rho(1)$ through $\rho(4)$.
2. For MA(1) with $\theta=-0.5$, calculate $\rho(1)$ and describe the visual pattern.
3. Factor $1-0.9z+0.2z^2$ and check stationarity.
4. Explain why an MA coefficient outside the invertible region is not necessarily an impossible data-generating process but is a non-unique parameterisation.
5. Write the companion matrix for the AR(2) in the worked root check and verify that its eigenvalues are $0.5$ and $0.25$.

Takeaway

ARMA is a disciplined approximation to stable linear dependence. The next module asks what that approximation implies for future values and uncertainty: Prediction.