Drift & retraining
A model trained on last year's data can quietly become less accurate as the world changes. Drift detection notices the change; retraining responds to it — through the exact same quality gate as the original model.
Drift, in plain language
The platform compares the distribution of the inputs arriving in production against the distribution the model was trained on, feature by feature, using a standard statistical score (PSI). Three bands:
| Score | Reading |
|---|---|
| Low | The world still looks like the training data. |
| Medium | Something is shifting — worth a look. |
| High | The model is seeing data it wasn't trained for. |
The headline drift score is the worst feature's score, not the average — an average would hide the one input that collapsed while nine stayed healthy.
Two honesty rules apply everywhere:
- Too little traffic → "skipped", never "no drift". A window with too few predictions is recorded as not evaluated; it is never reported as a reassuring zero.
- Stale evidence never triggers action. Only fresh, successful drift evaluations count.
You'll find drift status on each deployment's detail page — the latest run, per-feature scores, and the run history — with a button to evaluate on demand.
Retraining
Retraining re-runs the same pipeline you used the first time — train → evaluate → check acceptance → promote — on fresh data, with the same algorithm and settings as the current production model. It can be triggered three ways: manually, on a schedule, or by drift.
Set it up on the capability's detail page as a retraining policy: triggers, a cooldown between runs, and the approval mode.
The safety controls
Every control below is on by default or explicit — automation never gets a shortcut:
- The same gate. A retrained candidate faces the identical acceptance criteria. It cannot reach production by skipping the check.
- Human approval by default. A passing candidate parks as awaiting approval; a person compares it to the active model and decides. The acceptance check is re-run at approval time — never trusted from earlier.
- Better-than-current comparison. The candidate is compared against the live model on your first acceptance metric; matching the bar isn't enough if it's worse than what's already serving.
- A promotion budget. Automatic promotions are capped per day, so a misbehaving trigger cannot churn production.
- Cooldowns. A minimum interval between runs stops drift noise from retraining in a loop.
Rejected is not failed
A candidate marked rejected lost a gate — and production was untouched. That is the controls working, not an incident. Only failed (the pipeline itself errored) warrants investigation.
Next
- The gate it all flows through: Acceptance criteria.
- What it costs to run generative AI: Budgets & costs.