KPIs and Denominators
KPIs and Denominators
A metric is a contract
“Late-delivery rate” is undefined until all fields below are fixed:
| Field | HarborMart contract |
|---|---|
| numerator | accepted orders delivered after promised end |
| denominator | accepted orders with a completed delivery outcome |
| unit | order, not parcel or delivery event |
| period | promise date in the completed calendar week |
| timezone | Europe/London |
| exclusions | documented test orders; cancellations reported as a separate guardrail |
| revision | restate for late-arriving completion events for 14 days |
| owner | fulfilment analytics lead |
If 80 of 1,000 completed accepted orders are late, the rate is 8%. Dividing by 1,200 checkout attempts instead gives 6.7% and answers a different question.
Ratios do not average themselves
| Store | Late | Completed | Rate |
|---|---|---|---|
| A | 9 | 100 | 9% |
| B | 18 | 900 | 2% |
The unweighted average of store rates is 5.5%. The network rate is
Use the first only if the estimand is “equally weighted typical store.” Use the second for “share of all completed orders.” Weighting is part of the question, not a formatting choice.
Audit a pooled KPI
Driver trees expose arithmetic, not causality
Weekly contribution may be decomposed as
This identity helps locate a change. It does not prove that changing one component while holding the others fixed is feasible: a discount may raise conversion and lower margin simultaneously.
Outcome, driver and guardrail
| Role | Example | Why needed |
|---|---|---|
| outcome | contribution per eligible customer | links to value |
| driver | checkout conversion | reacts quickly |
| guardrail | refunds, lateness, complaints | detects displaced harm |
| process | picker minutes per order | diagnoses implementation |
Optimising a driver without guardrails invites gaming. Faster picking can raise errors; fewer refunds can reflect a harder claims process rather than better service.
Metric-change protocol
When a definition changes:
- version the formula and effective date;
- backfill only when the required historical fields exist;
- show old and new definitions during transition;
- explain the break rather than drawing a continuous trend;
- record affected decisions and dashboards.
Quick check
The late rate falls after orders without a valid delivered timestamp are excluded. What must be checked?