Data Provenance
Data Provenance
Every coefficient begins as a data lineage
For Pathways:
If an applicant ID is duplicated during linkage, the standard error and treatment mean may both change. “Data cleaning” is not neutral when it changes who represents the estimand.
Keep four layers separate
| Layer | Purpose | Rule |
|---|---|---|
| source/raw | immutable acquired files | never hand-edit |
| intermediate | typed, deduplicated, linked records | one scripted transformation per object |
| analytic | one row per declared unit/time | variables match the identification memo |
| output | tables, figures, model objects | generated, never manually corrected |
Store checksums or source version identifiers. Record access dates because live administrative extracts and web APIs change.
A useful data manifest
| Field | Pathways example |
|---|---|
| source owner | Northbridge scholarship office |
| file/version | lottery register, 2024-09-03 extract |
| unit and period | applicant, 2024 cohort |
| key | application ID; unique before linkage |
| coverage | submitted applications in participating districts |
| treatment time | offer issued by 31 August 2024 |
| outcome time | degree completion by 31 July 2029 |
| restrictions | de-identified secure access only |
| transformation | status codes mapped by versioned dictionary |
| known limitation | appeals after 31 August require separate flag |
This reveals a maturity problem immediately: five-year completion for the 2024 cohort is not observed in 2026. Use an earlier cohort, a shorter-horizon outcome or describe the analysis as prospective.
Joins need assertions
Before a merge, test key uniqueness and expected coverage. After it, report:
- rows before and after;
- unmatched keys on each side;
- one-to-one, one-to-many or many-to-many relationship;
- duplicated units created;
- match rates by treatment and outcome-relevant groups.
An overall 95% match rate can hide 99% among controls and 85% among treated units.
Time-stamp variables by causal role
| Variable | Measured | Role |
|---|---|---|
| prior score | before assignment | candidate confounder/precision variable |
| offer | assignment date | instrument or treatment |
| scholarship receipt | after offer | treatment uptake/mediator for ITT |
| first-year enrolment | after offer | outcome or mediator for later completion |
| completion | later | outcome |
The same field can be an outcome in one analysis and a mediator in another. A data dictionary should store definition, timing, coding, missing-value meaning and permitted role—not only column type.
Privacy and provenance reinforce each other
Keep direct identifiers outside the analysis repository. Document linkage without exposing keys, secrets or row-level personal data. For restricted sources, publish:
- data-access route and eligibility;
- non-disclosive metadata and variable construction;
- synthetic or simulated fixtures that exercise the pipeline;
- disclosure-control steps;
- code that runs once authorised data are mounted.
Quick check
The treatment indicator is reconstructed from the amount paid during the year. Which estimand is threatened?