Appendix and Revision Tools

Formula and Decision Map

A compact map from business questions to quantities, assumptions and actions

Formula and Decision Map

Describe what happened

Question	Quantity	Audit before use
What is typical?	$\bar{x}=\sum x_i/n$ ; median for the middle observation	skew, weights, repeated units, missingness
What share experienced an event?	$\hat p=y/n$	eligible denominator, complete follow-up, duplicate rows
How uncertain is a simple rate?	$SE(\hat p)\approx\sqrt{\hat p(1-\hat p)/n}$	independence and adequate counts; use better interval methods near 0 or 1
How did a KPI change?	$(x_1-x_0)/x_0$	same definition, period and currency; distinguish percent from percentage points
Which stage loses volume?	funnel conversion $n_{k+1}/n_k$	ordered eligibility and consistent cohort

Example: a late rate moving from 8% to 6% falls by 2 percentage points, or 25% relative. State which one.

Predict an unknown outcome

Question	Quantity	Decision interpretation
How large is numeric error?	$MAE=n^{-1}\sum	y_i-\hat y_i
Are large errors especially costly?	$RMSE=\sqrt{n^{-1}\sum(y_i-\hat y_i)^2}$	large misses receive quadratic weight
Are probabilities accurate?	Brier $=n^{-1}\sum(y_i-p_i)^2$	calibration and discrimination together
Does ranking separate events?	AUROC or precision–recall curve	ranking over thresholds, not policy value
Is a chosen action threshold worthwhile?	act if expected incremental benefit exceeds cost	requires calibrated risk and credible intervention effect

Always compare with a feasible baseline on data later than training. A lower test error is evidence about the defined population and period, not a permanent property.

Choose an action

Decision	Core object	Main assumption
uncertain alternatives	$EV(a)=\sum_s P(s)V(a,s)$	probabilities and consequences are adequate
value of perfect information	$EVPI=\sum_sP(s)\max_aV(a,s)-\max_aEV(a)$	perfect, costless and timely information benchmark
constrained allocation	$\max_x c^Tx$ subject to $Ax\le b$	objective, units and feasible set represent reality
one-period inventory	$F(Q^*)\ge C_u/(C_u+C_o)$	demand distribution and shortage/overage costs are stable
stable flow system	Little's Law $L=\lambda W$	consistent boundary and long-run averages
target an intervention	act if $\tau_iV_i-c_i>0$	$\tau_i$ is a credible individual or group treatment-effect estimate

Optimisation returns the best action inside the stated model. Test excluded constraints, alternative objectives and parameter ranges.

Estimate whether an action caused change

Design	Estimand or contrast	Key threat
randomised experiment	$E[Y(1)-Y(0)]$ under assignment	non-compliance, attrition, spillovers
difference in differences	$(Y_{T,after}-Y_{T,before})-(Y_{C,after}-Y_{C,before})$	non-parallel counterfactual trends
regression discontinuity	outcome jump at assignment cutoff	manipulation and extrapolation away from cutoff
adjusted observation	conditional treated–comparison contrast	unmeasured confounding and weak overlap

Prediction asks about $Y$ ; causal analysis asks about $Y(1)-Y(0)$ . A high predicted risk can coexist with zero treatment effect.

Operate the system

Layer	Numerator or signal	Denominator or reference
coverage	eligible cases scored on time	all eligible cases
adoption	executed recommendations	delivered or viewed recommendations—state which
calibration	observed event rate in a score band	mature, labelled decisions in that band
subgroup error	relevant errors for a defined group	actual negatives, positives or predictions according to metric
incremental value	outcome difference caused by policy	credible counterfactual, not historical total

A metric without its denominator, comparison and response rule is not operationally complete.

Next: Worked Examples and Pitfalls

Edit this pageorReport an issue

Appendix and Revision Tools

Formula routes, compact worked pitfalls and a dated evidence map

Worked Examples and Pitfalls

Short cases that expose common analytic errors before they reach a decision