Ten Worked Microeconometric Pitfalls
Ten Worked Microeconometric Pitfalls
1. Receipt replaces assignment
Offer is random; receipt is chosen. Regressing completion on receipt loses the experimental comparison. Estimate offer ITT, then use offer as an instrument for receipt if exclusion, monotonicity and relevance are defensible.
2. A mediator is called a control
Scholarship offer affects study hours, which affect completion. Controlling for study hours blocks part of the total effect and may induce selection. Decide whether the target is total, direct or mediated effect before adjustment.
3. Statistical significance replaces magnitude
An effect of 0.3 percentage points has in one million records. If programme cost requires at least 3 points, the estimate is precise evidence against practical value.
4. Cluster assignment, individual standard errors
Twenty schools are assigned treatment; 4,000 students are observed. The independent treatment variation is much closer to 20 clusters than 4,000 students. Cluster-aware inference and design limitations must be reported.
5. A strong first stage validates IV
Distance to an adviser strongly changes receipt but may also change mentoring directly. Relevance is strong; exclusion remains doubtful. Add an institutional pathway audit and exclusion sensitivity.
6. TWFE averages are treated as transparent
Early and late adopters have different dynamic effects. One TWFE coefficient uses already-treated comparisons. Estimate cohort-time effects, expose weights/support and aggregate toward a declared target.
7. An insignificant pre-trend proves DID
The pre-period interval allows trends from −4 to +5 points. Non-rejection has little evidential force. Report power/support and sensitivity to plausible post-treatment deviations.
8. RDD extrapolates to everyone
The cutoff effect is +9 points at score 70. Applicants scoring 55 differ in preparation and treatment response. A general expansion needs transport or new randomised evidence.
9. Matching creates randomisation
Observed covariates balance perfectly, but adviser motivation is unrecorded. Balance supports the implemented observable design; conditional exchangeability remains an assumption. Benchmark an unmeasured-confounding sensitivity analysis.
10. A predictive model discovers a causal policy
Prior attainment predicts completion but does not reveal who benefits from a scholarship. Estimate treatment-effect heterogeneity under a credible design and evaluate the policy rule on held-out data with capacity and fairness constraints.
Mixed-method challenge
A result has excellent pre-fit in synthetic control, a large post-gap, and no unusual placebo ratio after excluding all donors with pre-RMSPE above twice the treated unit’s value. What must be shown?