R Matrix Laboratory

R Matrix Lab 3 — Prediction and Gaussian Likelihood

Solve best-linear-prediction equations, verify Schur-complement uncertainty, and compare conditional, moment, and exact Gaussian AR estimates.

R Matrix Lab 3 — Prediction and Gaussian Likelihood

Question

How can one covariance matrix produce both a predictor and a likelihood?

Let the observed vector be $\mathbf X$ and target be $Y$ . Partition their joint covariance:

\operatorname{Var} \begin{pmatrix}\mathbf X\\Y\end{pmatrix} = \begin{bmatrix} \Gamma&\mathbf g\\ \mathbf g^\top&\gamma_Y \end{bmatrix}.

Then

\widehat Y=\mathbf a^\top\mathbf X,\qquad \Gamma\mathbf a=\mathbf g,

and

\operatorname{Var}(Y-\widehat Y) =\gamma_Y-\mathbf g^\top\Gamma^{-1}\mathbf g.

Example 1 — Predictor weights and Schur complement

Use an AR(2) with $\phi=(0.65,-0.20)^\top$ and innovation variance 1.5. Predict $X_{t+1}$ from

\mathbf X=(X_t,X_{t-1},X_{t-2})^\top.

The model says that only the first two coordinates should retain non-zero weights.

Solve and audit the best linear predictor

The third weight should be zero up to numerical precision, and the Schur complement should reproduce the innovation variance. Both are model-implied identities.

Example 2 — Three estimators answer three finite-sample problems

For a centred AR(1):

X_t=\phi X_{t-1}+\varepsilon_t.

Compare:

\widehat\phi_{CLS} =\frac{\sum_{t=2}^n x_tx_{t-1}} {\sum_{t=2}^n x_{t-1}^2}, \qquad \widehat\phi_{YW} =\frac{\widehat\gamma(1)}{\widehat\gamma(0)},

and the exact Gaussian maximum-likelihood estimate based on

\Sigma(\phi,\sigma^2) =\frac{\sigma^2}{1-\phi^2} \operatorname{Toeplitz}(1,\phi,\ldots,\phi^{n-1}).

Compare conditional, Yule-Walker, and exact Gaussian estimates

The estimates are not expected to match exactly:

Estimator	Initial-condition treatment	Main sample summary
conditional LS	conditions on $x_1$	transition residual sum of squares
Yule–Walker	replaces population moments	sample autocovariances
exact Gaussian ML	models stationary distribution of the full vector	determinant and quadratic form

Repeat with n <- 500. Differences usually shrink, illustrating asymptotic agreement without pretending finite-sample identity.

Numerical discipline

Do not write code that evaluates

det(Sigma)
solve(Sigma)

inside an optimiser. Determinants can underflow or overflow, and a full inverse performs unnecessary work. Use

U <- chol(Sigma)
logdet <- 2 * sum(log(diag(U)))
z <- backsolve(U, x, transpose = TRUE)
quadratic <- sum(z^2)

The code then mirrors the mathematics and exposes failure when $\Sigma$ is not positive definite.

Exercises

Predict $X_{t+1}$ for state $(1.2,0.4,-0.5)^\top$ by the AR(2) recursion and compare with the cell.
Remove $X_{t-2}$ from the predictor vector. Why does the MSPE remain unchanged under the true AR(2)?
Set $\phi_{\text{true}}=0.95$ and repeat the estimator comparison. Record convergence, condition number, and estimate dispersion across seeds.
Add an unknown mean $\mu$ to the likelihood and replace $\mathbf x$ by $\mathbf x-\mu\mathbf1$ . Which parameter correlations do you expect?
Graduate extension: use the delta method to transform the Hessian covariance from $\eta_1=\operatorname{atanh}(\phi)$ to $\phi$ .

Checkpoints

$0.65(1.2)-0.20(0.4)=0.70$ ; the third coordinate has no direct coefficient.
The Markov state already consists of $(X_t,X_{t-1})$ ; the older value adds no linear information conditional on that state.
Near the boundary, covariance matrices become ill-conditioned and normal approximations become less reliable.
The mean and persistence can be strongly confounded in short, persistent series.
Multiply the eta-scale variance by $(1-\phi^2)^2$ at the estimate.

Completion standard

You should be able to identify the covariance block used for prediction, verify its normal equations, and explain why conditional, moment, and exact likelihood estimates differ in finite samples.

Continue to Lab 4 — State Space.

Edit this pageorReport an issue

R Matrix Lab 2 — AR Recursions and Yule–Walker Equations

Convert AR models to companion form, compare polynomial roots with eigenvalues, propagate shocks, and recover coefficients from autocovariances.

R Matrix Lab 4 — State Space and Kalman Filtering

Implement Kalman prediction and correction from matrix equations, handle missing observations, and express AR forecasts as state propagation.