05 · Observation and intervention
Observation contributes to control when it can support an effective decision about the consequential process. The relevant evidence may include the proposed action, its authorization, the state already changed, and work accepted for later execution. A readable account of the assistant's reasoning can help explain a proposal while leaving those other facts unresolved.
Suppose a monitor flags a suspicious destination in the account workflow. Several possibilities remain: the proposal is still waiting; the change service has accepted a job; the address has changed; or a worker has already acted through the console. The alert has a different operational meaning in each state. Its usefulness depends on which state can be established and which action remains available.
Detection has a configuration#
I preserve the operational meaning of monitorability from AI Vision & Future: records must support reconstruction of relevant behavior and intervention on the workflow's timescale. Detection performance is one part of that requirement. A monitor's claim should name the behavior it is meant to detect, the evidence it receives, and how false alarms and missed deviations are measured. Measurement and monitorability
METR's preliminary monitorability evaluation supplies a bounded example. An agent is given a main task and a hidden side task; a monitor examines the visible record for the side-task behavior. The reported outcomes vary with the agent, monitor, and trace visibility at a configured false-positive threshold. The study has limited task diversity and elicitation. METR uses monitorability for how readily an AI system's behavior can be monitored. Its evaluation tests detection; the operational sense used here also requires records that support reconstruction and timely intervention. METR, Introduction, definition of monitorability; Methodology; Limitations
For the account service, the corresponding question is whether the monitor can distinguish an unauthorized destination from an unusual but legitimate recovery request. Flagging every unusual case might prevent some unattended mistakes while delaying the people who most need assistance. The evaluation therefore needs both missed deviations and legitimate work affected by the threshold.
Review capacity enters the same analysis. If a workflow permits more simultaneous cases than reviewers can assess before execution, the nominal review step may cease to provide the intended delay or scrutiny. This is a conditional relationship to test under the allowed workload. A low-volume demonstration supplies evidence for the workload it exercised.
Compare time from the same starting point#
For a reactive prevention claim, start the clock at the first observable sign that the incident has begun. Measure the interval until intervention becomes effective and compare it with the interval, from that same point, until the specified consequence becomes unavoidable. Detection, decision, queueing, delegation, and enforcement all consume time in the response path. Any earlier unobserved activity belongs in the estimate of total exposure.
An intervention that arrives early enough still needs to act on the correct process and cover the relevant paths. If an account change can proceed through either an accepted job or a worker's console, stopping one path leaves the other to examine. If the first sign arrives after private information has been disclosed, reactive prevention of that disclosure is already unavailable; intervention may still prevent further disclosure and support repair or recourse.
For accumulating effects, a single deadline is insufficient. The analysis must estimate what can happen before and during containment, including already accepted work. A claim that exposure stays below a limit needs evidence at the permitted action rate and concurrency. Average response time alone cannot establish a categorical bound for every in-scope incident.
The Knight Capital incident in Section 3 also shows how a response can worsen exposure. Before the market opened, automated emails reported an error described as “Power Peg disabled,” but they were not designed as alerts and were not acted on. During diagnosis, removing the new code from the seven correctly deployed servers activated the old code there too, worsening the incident. SEC Order 34-70694, ¶19, “BNET Reject E-mail Messages,” and ¶27, “Incident Response” This conventional case separates warning, intervention, and the accumulated effects described in Section 3; it supplies no AI response-time estimate.
A diagnosis must become a valid action#
Choosing an intervention introduces its own decision problem. Identifying a questionable recovery address still leaves the operator to choose an account, distinguish proposed from applied changes, and determine which jobs or access rights to suspend. The response can affect legitimate customers, so its authority and target also require justification.
In another account-service variant, repeated suspicious requests trigger an assistant's escalation and a suspension of recovery. An attacker submits those requests to keep a legitimate customer locked out. The path runs through the protective response itself. I assess who can trigger it, how suspension is authorized and lifted, and what recovery remains available to the customer.
R2Act examines a related distinction in a controlled microservice fault-recovery setting. It evaluates diagnosis, whether the selected operation and target fit the incident, and a replay result requiring both an offline-valid plan and restored service health. Strong diagnosis did not consistently yield valid recovery actions. The study uses injected faults in one testbed; it does not evaluate adversarial persistence, downstream repair, or organizational response. Its combined replay measure also does not estimate restoration for plans excluded by the validity gate. R2Act, §§V–VII, pp. 5–10
These observations support separate questions about detection, action selection, and resulting state. The cited studies do not form an end-to-end AI intervention experiment. Such a claim would require the chain to be exercised together under its declared operating conditions. That requirement carries observation and intervention into assurance: the evidence must cover the promise being made.