Data, Clocks, and Transformations

Transform prices and macroeconomic releases with explicit differencing, aggregation, timing, and vintage operators.

Data, Clocks, and Transformations

The transformation is part of the estimand

Write raw observations as $\mathbf z=(z_1,\ldots,z_T)^\top$ and analysed observations as

\mathbf y=M\mathbf z.

The matrix $M$ is not preprocessing trivia. It determines the units, timing, and economic meaning of every coefficient that follows.

Finance: from recorded price to investable return

For an adjusted price $P_t$ , the log return is

r_t=\log P_t-\log P_{t-1}.

If $\mathbf p=(\log P_1,\ldots,\log P_T)^\top$ , then

\mathbf r=D\mathbf p,\qquad D= \begin{bmatrix} -1&1&0&\cdots&0\\ 0&-1&1&\cdots&0\\ \vdots&&\ddots&\ddots&\vdots\\ 0&\cdots&0&-1&1 \end{bmatrix}.

Log returns add across time:

\sum_{j=1}^{h}r_{t+j}=\log(P_{t+h}/P_t).

Before using this identity, resolve:

splits and dividends: use a documented total-return or adjusted-price convention;
non-trading days: calendar time and trading time are not interchangeable;
bid–ask bounce and asynchronous prices: intraday observations can create mechanical serial dependence;
currency and timezone: a “daily” return must have a common valuation instant;
survivorship and delisting: today's constituent list cannot define yesterday's investable universe.

Arithmetic returns compound rather than add:

R_{t,t+h}=\prod_{j=1}^h(1+R_{t+j})-1.

Confusing sums with compounding is a units error, not a model-choice error.

Economics: levels, growth, gaps, and releases

For a positive level $Y_t$ , common targets include:

g_t^{(1)}=100\Delta\log Y_t, \qquad g_t^{(a)}=100m\Delta\log Y_t,

where $m=12$ for annualised monthly growth and $m=4$ for annualised quarterly growth. Year-over-year growth is instead

g_t^{(y/y)}=100(\log Y_t-\log Y_{t-m}).

These targets answer different questions. Annualising one-period growth rescales the latest pace; year-over-year growth compares levels a year apart and smooths short-run movement.

For macro data, record three dates:

reference period: the month or quarter being measured;
release date: when the estimate became public;
vintage date: which subsequently revised value is stored.

ALFRED preserves historical vintages, and the FRED real-time-period documentation explains why observation dates and real-time dates are separate. A real-time forecast must not use a later revision.

Aggregation is also a matrix

Suppose monthly growth is $\mathbf g_m$ . Quarterly log growth is obtained by summing non-overlapping blocks:

\mathbf g_q=A\mathbf g_m, \qquad A= \begin{bmatrix} 1&1&1&0&0&0&\cdots\\ 0&0&0&1&1&1&\cdots\\ \vdots&&&&&\ddots \end{bmatrix}.

Stocks and flows aggregate differently:

Variable	Economic type	Typical quarterly mapping
end-of-month interest rate	stock	last observation or average, depending on question
monthly sales	flow	sum
price index	index	average level, then compute change with stated convention
log growth	rate of change	sum for exact log change

There is no universal “convert monthly to quarterly” command.

R laboratory: make the operators visible

Price returns and monthly-to-quarterly aggregation

The three errors should be at machine precision. The important result is not the printed percentage; it is that the differencing and aggregation claims are explicit linear maps.

Worked choice: level, change, or gap?

Suppose the question is “Does the yield curve forecast recession?”

If the predictor is the ten-year minus three-month yield, the level difference is already an economically defined spread.
If the outcome is quarterly real GDP growth, use a documented growth convention.
If the outcome is a recession indicator, the model and loss must respect a binary event.
Differencing the spread merely because it is persistent changes the question from the curve's shape to the change in its shape.

The transformation should preserve the substantive signal the question names.

Transformation audit

Before estimation, write a one-row data contract for every variable:

Field	Required statement
raw source	provider, series, and access date
observation unit	currency, index, percentage points, or return
reference clock	calendar, trading, month, quarter
availability	release lag and publication timestamp
revision	first release, specified vintage, or latest revised
transformation	exact formula and order of operations
missing values	why absent and how treated
target horizon	one period, fixed $h$ , or cumulative/overlapping

Practice

Show that $D\mathbf 1=0$ . What information does differencing remove?
Replace the quarterly rows of $A$ by weights $(1/3,1/3,1/3)$ . Which variables would this suit?
A backtest formed in January uses a GDP value revised in June. Name the failure.

Answers

Every row contains $-1+1=0$ ; differencing removes a constant level and loses the initial condition.
A variable whose quarterly concept is an average stock or index, provided that convention matches the source.
Look-ahead through data revision: the June vintage was outside January's information set.

Next: Predictive Regressions and Persistent Predictors.

Edit this pageorReport an issue

Three Versions of Time Series

A decision guide to classical statistical, financial, and economic time series, with matched examples and method-selection rules.

Predictive Regressions and Persistent Predictors

Matrix derivations for forecasting regressions, overlapping horizons, persistent predictors, and Newey–West inference.