D10 — seen, traced, and stopped

I have sat in more than one incident review where the most honest sentence was that nobody had noticed.

Not nobody had cared. Nobody had noticed. The system had drifted for days, or an agent had been quietly doing the wrong thing across a few thousand interactions, or a model had started refusing a class of request it used to answer, and the first signal that reached a human was a customer complaint, a finance query, or a journalist. By the time the room assembled, the question was no longer how to detect the problem. It was how far back to apologise.

That is the failure D10 is built to prevent. The earlier dimensions in this pillar ask whether the right controls exist. D10 asks a colder question.

Can you see the system while it is still moving?

That is the work of D10, Observability and Incident Response, in the framework. It is the easiest dimension to mistake for tooling — a dashboard, a logging stack, a paging rotation — and the hardest to fake under pressure. An adaptive system has a property that a fixed one does not: it changes between releases, it drifts between days, and it acts through tools and vendors the enterprise does not fully own. You cannot govern that with a quarterly report. The governing premise of this dimension is plain. Observability is the precondition for accountability. You cannot be accountable for a system you cannot see, cannot trace, and cannot stop.

The differentiator is not any single monitor. It is whether four things hold together at the speed the system changes: the estate can be seen, a misbehaving instance can be traced to a decision and a cause, the harm can be stopped on a named trigger, and the stop teaches the next release. Most organisations have fragments of the first. Very few have wired the fourth back into the first.

D10.1 is the floor: agent telemetry and cost observability. Calls, tool invocations, latencies, token spend, error and refusal rates, retry storms, and the per-workflow cost of running an agent that loops more than anyone modelled. This sounds like an SRE concern, and operationally it is — the buyer owners here are the CTO, platform and site-reliability operations, and the CISO. But the operating-model point is not that the telemetry exists. It is that it is attributed. A cost line that cannot be traced to a use case, an owner, and a decision class is a number waiting for an argument. A latency or refusal spike that nobody connects to a model change is drift that has already happened. Mature telemetry is joined to the D2.3 criticality of the workflow and the D5 autonomy and blast radius of the agent, so that the same spike on a low-impact internal assistant and a customer-facing decision agent are not treated as the same event.

D10.2 is where the adaptive nature of the system bites: drift, quality, and hallucination monitoring. A traditional service degrades visibly — it errors, it slows, it falls over. An AI system can degrade while every metric stays green. It answers fluently and wrongly. It drifts as the world it was tuned against moves underneath it. A vendor ships a new model family behind the same API name and the behaviour shifts without a deployment on your side. The hard discipline is not buying a drift product. It is deciding, per use case, what “good” means in evidence — which quality signals are tracked, what a meaningful regression looks like, how hallucination or unsupported-claim rates are sampled and reviewed, and which of those signals is allowed to trip an alarm rather than fill a report nobody reads. Where the methodology gets genuinely technical — statistical drift detection, evaluation design, the sampling that makes a hallucination rate defensible — that is specialist work. Model-risk, data-science, and evaluation specialists determine the method. The operating-model question is narrower and non-negotiable: does the signal exist, is it owned, and can it change a decision?

D10.3 is the hinge of the whole dimension, and the place the soft answers collect: anomaly detection, alerting, and the stop. It is comfortable to talk about detection. It is uncomfortable, and necessary, to say who is allowed to halt a production AI system, on what trigger, without waiting for a meeting.

This is where “integrate into existing committees” becomes too soft unless it is paired with hard accountability. A monitoring capability that can only escalate into a forum is not an incident control; it is a notification service. The maturity signal is a named, pre-authorised stop. For each material AI workflow, someone — an on-call operator, a control owner, a second-line risk function — must hold the authority to suspend, throttle, roll back, or revoke the agent’s access on a defined trigger, before the accountable executive is even reached. The trigger has to be specified in advance: a drift threshold breached, a refusal or error rate beyond tolerance, a tool-access anomaly, a confirmed harmful output, a vendor incident notice. The mechanism has to be tested, because a kill switch that has never been pulled is a belief, not a control. And the authority has to connect cleanly to D6 decision rights, so that the person empowered to stop the system in the moment, and the executive accountable for it afterward, are not strangers to each other.

I am deliberately not prescribing who that person is. The framework names the accountable outcome, not a title. The stop authority can sit with a CAIO, with an extended CIO, CDO, or CTO mandate, with a chief risk officer or operational-resilience executive under a three-lines model, or with CEO-direct sponsorship for the most material cases. What is not optional is that the authority is named, pre-authorised, time-bounded, and rehearsed. The dangerous pattern is the implicit one — everyone assumes someone could stop it, and in the incident nobody is sure they are allowed to.

D10.4 is the incident path itself, and this is where the Australian anchor becomes concrete. APRA’s CPS 230, Operational Risk Management, has been in force since 1 July 2025 for APRA-regulated entities. It is an operational-risk and resilience standard — critical operations, disruption tolerances, material service-provider risk, and incident management raised to a board-level discipline. It is worth being precise about what CPS 230 is not. It is not an AI law, and it is not an information-security standard; the security perimeter belongs to CPS 234 and lives in the neighbouring dimension. D10 sits squarely on the operational-risk side: when an AI workflow is part of, or material to, a critical operation, its incidents are operational-risk incidents, and the existing CPS 230 machinery is where they belong. Specialist regulatory counsel determines applicability, tolerance levels, and notification timing for the entity. The operating-model obligation is to make the AI incident legible to that machinery before the incident, not to invent a parallel one beside it.

That last point is the discipline. The wrong instinct, in 2026, is to stand up a freestanding “AI incident” process with its own runbook, its own forum, and its own clock. In an APRA-regulated organisation that reads as politically naive — a new bureaucracy competing with the operational-risk function that already owns critical-operation incidents and already answers to the board for them. The mature pattern is the opposite: AI incidents are classified into the operational-risk taxonomy the organisation already runs, severity-rated against the same disruption tolerances, and escalated through the same channels, with AI-specific evidence attached. The detect-contain-notify-learn path is one path. AI just changes what containment looks like (a model rollback, an access revocation, a fallback to a human queue) and what the evidence package must hold (which model or agent version, the prompt and tool-call trace, the affected decisions, the scope of harm).

Notification is where most of the loose thinking lives, and where deferral is mandatory. There is no AI-specific Australian notification scheme. AI incidents map onto the schemes already in place where their preconditions are met. If personal information is involved, the OAIC Notifiable Data Breaches scheme under the Privacy Act 1988 may engage, and privacy counsel determines eligibility and timing — the scheme runs on its own awareness-and-assessment logic, not a borrowed seventy-two-hour clock. For APRA-regulated entities, CPS 230 material-incident notification considerations may apply, determined by regulatory counsel. A listed entity may face continuous-disclosure obligations under ASX Listing Rule 3.1 and Corporations Act section 674 if the event is price-sensitive, determined by corporate counsel. A Commonwealth entity follows different pathways again. The operating-model job is not to know which clock fires; it is to ensure the incident path can produce the evidence each of those specialists needs, fast enough that the binding clock — whichever one it turns out to be — is met from facts and not from reconstruction.

D10.5 closes the loop that separates an estate that learns from one that merely survives: post-incident review and continuous learning. The review is not a retrospective filed and forgotten. Its output has to land back in the system that produced the incident — a tightened evaluation gate in D5, a refreshed AI impact assessment in D7, a new drift threshold in D10.2, a revised stop trigger in D10.3, a vendor incident-SLA renegotiation that D8.4 then enforces. The maturity test is whether the lesson changed a control, and whether the next occurrence of the same failure class was caught earlier, or did not occur. A post-incident review that does not move a threshold, a gate, or an owner is a document. An estate that keeps relearning the same lesson does not have continuous learning; it has a recurring meeting.

This is where the Responsible and Agentic AI Governance pillar lands its quiet argument. The earlier dimensions design the controls. D10 is the dimension that finds out whether they were ever real, because it watches the system in production, under load, while it changes. A control that has never been tested against a live failure is a hypothesis. Observability is how the hypothesis gets tested without a customer being the test.

So the L4 buyer question is deliberately unkind. Take a single material AI workflow and show, for one real or simulated incident in the rolling twelve months: the telemetry that detected it, the trace from symptom to model version and decision, the named person who held the stop authority and the trigger they acted on, the operational-risk classification it was filed under, the specialist-determined notification assessment, and the control that changed afterward — all reconciled to the same event. Without that reconciliation, an organisation can show a handsome dashboard, a thick runbook, and a forum with minutes, and still not be able to prove it would have seen the thing in time.

The test I would actually apply is smaller and harder to dress up. Pick one production agent and ask three questions in order. If it started behaving wrongly right now, what would tell you, and how soon? Who is allowed to stop it before the next forum, and on what trigger? And the last time something like this happened, what control is different today because of it?

If those three answers exist and connect, D10 is doing its job: the system can be seen, traced, and stopped at the speed it changes. If they do not, the organisation does not have AI incident response. It has a plan to start paying attention after the apology.