3. Policy Evaluation Designs

Staggered DID and Event Studies

Estimate cohort-time effects without contaminated comparisons and test sensitivity to trend violations

Staggered DID and Event Studies

One treatment coefficient can mix incompatible comparisons

Suppose municipalities first adopt in 2023, 2024 or never. A conventional two-way-fixed-effects regression may compare:

newly treated with never treated;
newly treated with not-yet treated;
later treated with already treated.

The last comparison is problematic when effects evolve: an already-treated unit is not an untreated counterfactual. Heterogeneous effects can produce opaque or negative weights.

Build group-time effects first

Let $G=g$ denote first treatment in period $g$ . Define

ATT(g,t)=E[Y_t(g)-Y_t(0)\mid G=g].

Example estimates:

Cohort	Impact year	One year after	Two years after
2023 adopters	−4	−7	−9
2024 adopters	−3	−5	not mature

An overall ATT, a cohort average and an event-time average use different weights. Report the weighting rule and keep support visible.

Three modern routes

Route	Core construction	Question to audit
group-time ATT	compare each cohort with never/not-yet treated	which control group and conditional trend?
interaction-weighted event study	estimate cohort-specific dynamics, then aggregate	which cohorts support each horizon?
imputation	estimate untreated outcome model using untreated observations	where is $Y(0)$ extrapolated?

Callaway and Sant’Anna (2021) formalise group-time effects and aggregation. Borusyak, Jaravel and Spiess (2024) derive a robust and efficient imputation approach under stated event-study assumptions. These are related solutions, not interchangeable commands.

Event time can change composition

At event time +3, only early adopters have three post-treatment periods. A falling event-study line may reflect both dynamics and changing cohort composition.

Report:

number of cohorts and observations at each horizon;
balanced-horizon estimates where useful;
cohort-specific results;
whether long-run outcomes are mature;
simultaneous rather than only pointwise uncertainty for a plotted path.

Pre-trends are evidence, not proof

Failure to reject pre-treatment coefficients can mean parallel trends or low power. Rejecting can reflect true differential trends, anticipation, outcome noise or contaminated estimation.

Rambachan and Roth (2023) replace exact parallel trends with restrictions on how post-treatment violations relate to pre-trend differences. The useful output is a sensitivity statement:

“The negative effect remains below zero if the post-policy trend deviation is no more than 1.5 times the largest pre-policy deviation.”

The multiplier must be justified substantively; it is not a universal threshold.

Staggered-design checklist

define treatment adoption, reversals and anticipation;
draw cohort by calendar time;
state $ATT(g,t)$ and desired aggregation;
identify never-treated or not-yet-treated comparisons;
estimate only supported cohort-time cells;
show pre-period evidence and trend sensitivity;
cluster/infer at the assignment level;
distinguish event dynamics from cohort composition.

The official did package documentation provides a current implementation of group-time effects, event aggregation and simultaneous bands. Software defaults still require a defensible control group and anticipation rule.

Quick check

At event time +5, the estimate uses only one early-adopting cohort. Can it be called the general five-year policy effect?

Answer

Not without a transport argument. It is a five-year effect for the surviving early cohort under its treatment version and context. Show support, compare cohorts at shorter common horizons and limit the claim.

Next: Regression Discontinuity

Edit this pageorReport an issue

Difference-in-Differences

Use a comparison group’s change to estimate the treated group’s missing untreated trend

Regression Discontinuity

Turn a treatment threshold into a local causal comparison and audit continuity, manipulation and bandwidth