In most enterprise AI deployments, “human in the loop” is treated as a safety guarantee. Put a reviewer in front of a probabilistic system and its output is assumed to become accountable: the model may err, but the human will catch it.
A human reviewer is not an independent control plane sitting outside the system. Reviewers operate inside the same delegation structure, under the same throughput pressure, incentives, partial information, and local optimization. At scale, human override is not a check on authority. It is another authority surface, one that expands scope, normalizes exceptions, and quietly redefines what the institution permits.
Override is not governance. Mistaking it for governance is how authority expands while everyone believes it is being contained.
Instance Correction Is Not Boundary Design
Override and governance operate at different levels of the system, and the difference is the whole point.
Override acts on individual decisions. A model recommends approval; a reviewer rejects it. A case is escalated; a classification is revised. The bad output is caught, the questionable action halted, the instance handled.
Governance acts on boundaries: the set of actions the system is permitted to take in the first place. In control terms it operates on reachable transitions, the actions the system is authorized to execute under defined conditions. Correcting an instance changes an outcome. It does not change a transition.
So repeated correction does not accumulate into control. When correction stays at the level of the instance, the delegation structure underneath is untouched. The same recommendation returns tomorrow, the same escalation next week, and each time the authority that produced it is still there. A system that constantly requires human correction has not been constrained. It has been accommodated: the institution absorbs the error pattern instead of narrowing the authority that generates it.
Override operates on cases. Governance operates on risk classes, the categories of action authorized under stated conditions. Correct outputs without constraining the transitions that produce them and you are not governing the system. You are absorbing its friction.
Why Override Is a Drift Vector
Governance runs on two risk postures, and override is the gap between them.
Approved risk posture is what the institution ratified: the documented appetite, the tier definitions, the exposure limits, and the delegation thresholds endorsed in committee and written into governance artifacts. It states which actions are authorized, under what conditions, and at what level of accountability.
Applied risk posture is what the system actually permits under load: the decisions executed without escalation, the exceptions routinely tolerated, the workflows people fall back on when pressure rises. Approved posture is written in policy. Applied posture is written in pattern.
In stable systems the two stay close. Under scale they separate, and the separation starts at the boundary, not in a policy document. Override is where it first becomes visible. A recommendation nears a threshold; a reviewer intervenes. Once is a correction. The same intervention recurring in the same risk class, the same edge case approved, the same escalation invoked, the same category reinterpreted, is not correction. It is a standing operating pattern the institution never authorized.
Read override as the signal it is. Repeated override in a risk class means the delegation surface for that class is mis-specified. A sustained rise under load means throughput pressure is rewriting how authority is exercised. Escalation thresholds that soften to preserve velocity mean applied posture has already moved while approved posture has not. None of this requires a policy change. It requires only consistent adjustment under pressure.
Governance drift is not a policy event. It begins when applied posture diverges from approved posture and nothing contracts the delegation surface in response. Unmeasured override is exactly that: drift with no corrective attached.
Propagation, Not Intent
This is not a human problem. It is a property of any system that adjusts at the edge without contracting the boundary: the pattern reinforces itself.
The AI version usually gets moralized. Bias is described as a property of the model, evidence of bad intent or defective judgment. That is imprecise. In a probabilistic system, most of what gets called bias is statistical skew amplified through feedback: outputs shape later inputs, and the distribution reinforces itself faster than correction can catch it.
Human workflow does the same thing. When override absorbs the same exception without changing the surface that generates it, decisions near the boundary shift in one direction, escalations recur in the same class, informal readings harden into routine, and applied posture bends to match. The model’s distribution shifts through data; the institution’s shifts through workflow. Neither moves in a single decisive step. The boundary is not rewritten. It is reinterpreted until interpretation becomes practice.
Propagation at scale does not announce itself with failure. It shows up as pattern: repetition, clustering, normalization at the edge. And patterns, unlike intentions, can be measured.
Override as Telemetry
Override is telemetry, not reassurance. Its value is not in the count.
Frequency alone says little: a low rate does not prove stability, a high rate does not prove vigilance. What matters is distribution, trajectory, and the conditions under which override happens. So instrument it. Capture every override as an event tagged with what makes it legible: the risk class, the decision boundary it sits against, the reviewer, the operating load at the time, and the escalation latency it took to resolve. Read it by segment, not in aggregate.
Two signals matter most.
Clustering. Override concentrating on one decision boundary inside a risk class, the same recommendation adjusted again and again, the same escalation invoked with rising regularity, is a sequence of judgments resolving into an operating pattern. The governance artifact has not changed. The applied posture has.
Threshold softening under load. This one hides even when frequency looks flat. As pressure rises, escalation latency compresses, and decisions that once triggered formal review get resolved through expedited interpretation. The policy language is intact; the system’s tolerance for exposure has widened.
Neither appears in summary reporting. Both appear in segmented, time-series override data. Read that way, override is the earliest indicator that a delegation surface is under strain, and it names the risk class compensating for friction instead of constraining the authority that produces it.
Governing the Delegation Surface
Telemetry only matters if something acts on it. Override that is measured but never changes authority is accommodation with better dashboards. Governance is the mechanism that contracts the delegation surface when the telemetry says applied posture has moved. In practice that means four things.
Bind override signals to a trigger. For each risk class, define the condition that forces a delegation review: override clustering past a set rate, a sustained rise under load, escalation latency falling below a floor. The trigger is a control, not a report. It fires on its own; it does not wait for someone to notice.
Make the corrective a contraction of authority, not more review. When a class trips its trigger, narrow the reachable transitions in that class: reduce what runs without escalation, reclassify the action to a higher tier, or remove the transition entirely. Adding another reviewer widens the surface. Narrowing it is the point.
Give the boundary an owner and a gate. Instance correction belongs to reviewers. Boundary design belongs to governance. Route the override telemetry from the reviewers to the owner of the risk class, who must either contract the surface or record, explicitly, why the divergence is now the approved posture. Silent tolerance is not an option the mechanism allows.
Reconcile applied to approved on a cadence. On a fixed interval, compare what each risk class actually permitted against what was ratified. Where they diverge, one of two things happens: the surface contracts back to the approved posture, or the approved posture is formally amended with accountability attached. Divergence is never allowed to persist unnamed.
Closing
Human judgment is an input, not an architecture. Under scale it is one input among many, shaped by incentives, pressure, and incomplete information. A control structure that assumes those inputs will keep resolving ambiguity in the right direction is not a control structure. It is a hope written into the workflow.
Override resolves cases. Governance defines and contracts the risk classes those cases come from. Rely on repeated intervention instead of structural change and authority keeps expanding while the outputs look corrected.
An unconstrained delegation surface does not stay neutral. It adapts to the pressure on it. So governance cannot end at review. It has to own the mechanism that withdraws authority when drift appears: the trigger that fires, the review it forces, the reachable transitions it removes. With that mechanism, override is an instrument. Without it, override is accommodation, and drift is the operating model.