Repair is not another word for fine-tuning. It is a controlled process for changing the smallest supported part of a deployed model while preserving a record of why the change was proposed and why it was accepted.
Detect before changing
The system first needs evidence that the current model is no longer competent for the observed machine. Detection should use public telemetry and command history rather than hidden simulator state.
The detector must also be evaluated on unchanged machines. A mechanism that catches every fault by constantly triggering is not useful.
Compile a bounded candidate
A repair candidate names the affected mechanism, parent model, evidence, allowed parameter or transform and rollback path. The local form makes the proposed change easier to inspect and compare with broad retraining.
When the evidence supports more than one explanation, the correct outcome is a question or an abstention rather than an invented correction.
The model cannot approve itself
A separate verifier checks changed-regime improvement, old-regime retention, constraints, provenance and artifact identity. A failed check leaves the active version unchanged.
This boundary is central to A08. It turns adaptation into a versioned engineering decision rather than a silent parameter update.
Measure the mission
Prediction error matters, but the final test is whether the repaired model restores the decision or mission it is intended to support. Success, failure, false triggers, abstentions and violations should be reported separately.
A progress score is not mission success. A robot that approached a target but did not satisfy the registered terminal condition failed that mission.
Sources
NIST AI RMF: Measure ↗Guidance on measurement, evaluation and documented limitations.
Model Cards for Model Reporting ↗A framework for documenting intended use, evaluation context and limitations.