4. Deployment and Governance
Operate an analytic decision system with monitoring, accountability and value evidence
4. Deployment and Governance
A notebook ends when the result is produced. A decision system begins when the result can affect people, money or operations repeatedly.
Every arrow is a possible failure point. Monitoring only model accuracy is therefore insufficient.
Learning outcomes
After this module, you should be able to:
- specify a data product and decision-policy interface;
- monitor service, data, model, decision and outcome behaviour separately;
- design shadow tests, staged rollout, rollback and incident review;
- audit fairness, privacy, security, human review and contestability;
- distinguish technical adoption from causally demonstrated business value.
The operating question
| Layer | Question | HarborMart evidence |
|---|---|---|
| service | Did the system respond correctly and on time? | latency, failed requests, stale features |
| data | Do inputs still mean what the contract says? | schema, missingness, timestamp and range checks |
| model | Do scores still rank and calibrate? | later labelled outcomes by period and group |
| policy | Were scores converted into the intended actions? | threshold, capacity rule and override log |
| outcome | Did the action improve the objective without unacceptable harm? | experiment or credible comparison |
The owner and response rule must be written beside every alert. An unattended dashboard is not a control.
Chapter route
- Data Products and Monitoring — contracts, drift, feedback and rollback.
- Governance, Fairness and Privacy — accountability across the lifecycle.
- Adoption and Business Value — test whether the deployed policy changes behaviour and value.
Prepare, work, follow up
- Prepare (45 min): draw the complete path from an order event to a capacity action. Mark every timestamp and owner.
- Workshop (120 min): run a simulated incident review: stale capacity data makes the late-delivery score understate risk during a storm.
- Follow up (60 min): submit a one-page control plan containing five monitors, alert thresholds, owners, rollback criteria and one value test.
Postgraduate extension: treat the policy as an adaptive intervention. Explain how selective labels, feedback loops and changing treatment effects invalidate a static evaluation.