Governance, Fairness and Privacy
Governance, Fairness and Privacy
Governance begins before modelling
“The data science team owns the model” is incomplete. A defensible system names:
- a decision owner accountable for the policy;
- a data owner accountable for meaning, access and retention;
- a technical owner accountable for implementation and monitoring;
- an independent reviewer able to challenge evidence and controls;
- an affected person with a usable explanation and route to correction where appropriate.
Responsibility cannot be outsourced to a metric or vendor.
Use frameworks as questions
The voluntary NIST AI Risk Management Framework organises work into four functions:
| Function | HarborMart question | Evidence |
|---|---|---|
| govern | Who can approve, change or stop the delivery policy? | roles, inventory, review cadence |
| map | Which people, contexts and harms can the policy affect? | use-case boundary and impact map |
| measure | How will validity, fairness, privacy and safety be tested? | test protocol and results by group |
| manage | Which risks are accepted, mitigated, transferred or avoided? | controls, owner, deadline and residual risk |
NIST states that AI RMF 1.0 is being revised, so cite the version and review date. ISO/IEC 42001:2023 takes a management-system approach: policies, objectives, processes and continual improvement across an organisation. Neither framework turns a harmful objective into a good one; both require substantive judgement.
Fairness needs a decision and a harm
Suppose the system flags customers for manual delivery-window review. On a test set of 200 customers in each group:
| Group | TP | FN | FP | TN | TPR | FPR | PPV |
|---|---|---|---|---|---|---|---|
| A | 36 | 4 | 16 | 144 | 90.0% | 10.0% | 69.2% |
| B | 27 | 13 | 8 | 152 | 67.5% | 5.0% | 77.1% |
Both groups have about 90% accuracy, yet Group B experiences more missed late deliveries. Group A experiences more unnecessary reviews. No single column defines fairness because the harms differ.
Audit in this order:
- meaning: are groups and outcomes measured comparably?
- representation: who is absent, rare or poorly labelled?
- performance: calibration and relevant errors with uncertainty;
- allocation: who receives benefits, burdens and human review?
- recourse: can an affected person correct data or challenge an action?
- repair: change data, objective, workflow, threshold or product boundary—and retest consequences.
Small groups produce unstable rates. Report counts and intervals, not only percentages. A group-specific threshold may create new legal or ethical problems; obtain domain and legal review rather than treating it as an automatic technical fix.
Privacy by necessity
For each field, ask “what decision-relevant purpose requires this value?”
| Proposed field | Tempting rationale | Better control |
|---|---|---|
| exact date of birth | “might improve prediction” | use a justified age band or omit it |
| full postcode | captures distance | derive coarse distance, then restrict or delete source |
| free-text service notes | rich context | exclude by default; may contain sensitive or unsafe text |
| customer identifier | joins outcomes | replace with controlled pseudonymous key |
Minimisation reduces breach impact and spurious proxy use. It does not by itself make processing lawful. Specify purpose, lawful basis where relevant, access, retention, deletion, provenance and permitted downstream use with appropriate experts.
Security is part of analytic validity
Threats include altered source data, stolen credentials, poisoned labels, model extraction, prompt or input injection in AI-assisted workflows, and unauthorised policy changes. Controls should cover least privilege, secrets management, integrity checks, dependency review, immutable logs and tested incident response.
A perfectly calibrated model fed manipulated capacity data is an unsafe decision system.
Human review must be designed
“A human is in the loop” is not a safeguard unless the person has:
- time and information to disagree;
- authority to change the action;
- guidance for difficult cases;
- protection from automation bias and rubber-stamping incentives;
- an override log reviewed for systematic patterns.
For a delivery disruption, show the reason, material evidence, alternatives and uncertainty—not an unexplained risk score.
A dated legal boundary
The original EU AI Act, Regulation (EU) 2024/1689, was amended by Regulation (EU) 2026/1744. The European Commission's 31 July 2026 update reports that the amendment entered into force on 27 July 2026 and changed important high-risk-system timelines.
Therefore, never label a system or deadline from memory. Identify jurisdiction, role, intended purpose, affected people and the provision in force on the review date. This course offers analytical governance, not legal advice.
Quick check
HarborMart removes the customer-group field from training. Has it solved unfairness?