0. Causal Questions and Designs

Causal Questions and Estimands

Translate a policy question into treatment versions, potential outcomes and a target population

Causal Questions and Estimands

Replace the topic with a contrast

“Does university help earnings?” leaves every important object open. A researchable version is:

Among Pathways applicants scoring 68–72 in 2024, what is the five-year effect of being offered the stated scholarship package, compared with no offer, on degree completion?

Element	Why it matters
population	effects may differ outside applicants or the score band
treatment	offer is not receipt, enrolment or completion
comparator	“no offer” may still include other financial aid
outcome	entry, completion and earnings answer different questions
horizon	an early entry effect can fade or grow
treatment version	amount, duration and advising may vary

Potential outcomes name the missing comparison

Let $Y_i(1)$ be completion if applicant $i$ is offered Pathways and $Y_i(0)$ completion without the offer. The individual effect

\tau_i=Y_i(1)-Y_i(0)

cannot be observed because only one treatment state occurs. The observed outcome is

Y_i=D_iY_i(1)+(1-D_i)Y_i(0).

Econometric design supplies information about the missing mean, not the missing individual outcome.

Choose the population before the acronym

Estimand	Definition	Policy interpretation
ATE	$E[Y(1)-Y(0)]$	average effect in the stated population
ATT	$E[Y(1)-Y(0)\mid D=1]$	effect for those treated
ATU	$E[Y(1)-Y(0)\mid D=0]$	effect for those currently untreated
ITT	effect of random assignment or offer	effect of the policy lever actually randomised
CATE	$E[Y(1)-Y(0)\mid X=x]$	effect for a pre-defined covariate profile

These quantities coincide only under special conditions. A voluntary programme’s ATT can differ from the effect of expanding it to current non-participants.

The treatment ladder

For Pathways:

\text{eligibility}\rightarrow\text{offer}\rightarrow\text{receipt} \rightarrow\text{enrolment}\rightarrow\text{completion}.

An eligibility lottery identifies the ITT of an offer. Estimating the effect of enrolment requires more assumptions because applicants can decline an offer or enrol without it. Never switch the treatment label after seeing which effect is larger.

Consistency and treatment versions

Consistency says the observed outcome under the received treatment equals the corresponding potential outcome. This becomes doubtful if “scholarship” mixes £1,000 and £8,000 awards, one-year and four-year support, or advising and no advising.

Repair the question by:

defining the intervention precisely;
recording implementation and exposure;
estimating version-specific effects where support permits;
limiting the claim when versions cannot be separated.

Interference changes the unit

If scholarships fill a fixed number of university places, one applicant’s offer changes another applicant’s outcome. Then $Y_i$ may depend on the vector of offers, not only $D_i$ .

Possible targets include a direct effect at a stated saturation, a school-level policy effect or a total equilibrium effect. “Robust standard errors” do not repair an incorrectly defined potential-outcome structure.

A one-minute estimand test

Complete this sentence without method names:

For population, compare the mean outcome at horizon under treatment version with the mean under comparator.

If the sentence cannot be completed, a regression formula is premature.

Quick check

A study randomises scholarship offers but reports the effect of university attendance by comparing attenders with non-attenders. Which estimand is directly identified by randomisation?

Answer

The ITT effect of the offer. Attendance is post-assignment and selected. The offer may be used as an instrument for attendance only with additional relevance, exclusion, assignment and monotonicity assumptions; the resulting LATE applies to compliers, not automatically to all students.

Next: Potential Outcomes and Experiments

Edit this pageorReport an issue

0. Causal Questions and Designs

Define the counterfactual target before selecting data, controls or estimators

Potential Outcomes and Experiments

Use assignment to recover an average counterfactual while auditing non-compliance and missing outcomes