Decision before model
A prediction creates no value if nobody can act on it, the action is unavailable, or feedback is not captured. The work begins with the decision, available interventions, cost of errors, timing, and accountable owner.
Evaluation that reflects consequence
- Incumbent process or simple baseline
- Out-of-time holdout
- Calibration and uncertainty
- False-positive and false-negative cost
- Subgroup and context performance
- Shadow and limited-live outcomes
- Operational latency, reliability, and cost
Production discipline
The controlled baseline includes data, features, model version, thresholds, explanations, integration, user interface, monitoring, fallback, release criteria, and change history. A provider update is treated as a behavioral change until evaluated.
Stop conditions
- No meaningful improvement over the incumbent.
- Predictions cannot trigger an effective intervention.
- Outcome feedback is not captured.
- Drift or subgroup performance cannot be monitored.
- Users systematically override the output for valid reasons.
- Operating cost exceeds incremental decision value.
