AI Agent Incident Response: A Production Runbook

That sequence is familiar to reliability engineers, but agents add a different kind of risk. A normal service can return an error and stop. An agent can call a tool, change external state, continue to another step, and only then expose the failure. By the time an error reaches the operator, the important question may no longer be what failed. It may be what the system already changed.

Google SRE incident response guidance puts mitigation before deep root cause analysis. The practical sequence is to assess impact, reduce the damage, investigate the cause, apply the fix, and document what happened. For an agent system, mitigation means controlling action authority as well as traffic.

The NIST AI Risk Management Framework makes the same operational point from a risk perspective. Its manage function calls for processes that track, respond to, and recover from AI incidents or errors. It also expects mechanisms that can disengage or deactivate an AI system when outcomes no longer match intended use.

That gives you a useful boundary for the runbook. Treat every incident as two linked problems: restore technical service, and restore confidence that the agent will take only intended actions.

A useful incident commander keeps those two tracks separate. The technical track asks whether the service can run again. The action safety track asks whether the agent can act again without repeating or extending the original failure. Restoring one before the other is how a contained outage turns into a second incident.

Agent incidents are easy to miss when monitoring stops at uptime, latency, and error rate. Those signals matter, but they describe the service envelope. They do not tell you whether the agent chose the right tool, followed the right path, or returned an acceptable result.

OpenTelemetry's GenAI semantic conventions define agent invocation spans with fields for the agent, conversation, operation, and error type. That trace vocabulary gives you the execution story: which agent ran, what operation it attempted, where an error surfaced, and how the call sits inside the wider trace.

Output quality needs a second signal. OpenTelemetry also defines a GenAI evaluation result event for recording evaluation outcomes such as quality or accuracy. That matters because a tool call can succeed, the model can return a response, and the incident can still be real. A silent quality regression is operationally different from a timeout, but it can hurt users just as much.

The LLM observability guide is the useful companion here. The incident view should let you move from a bad user outcome to the exact agent invocation, tool call, state transition, and evaluation result that produced it.

A practical detection rule follows from those two telemetry layers. Page on hard operational failures when they threaten service or action safety. Route quality failures into an evaluation based alert or review path. Do not force both through the same threshold just because they share the same agent.

Containment means reducing what the system can still do while you learn what happened. For an agent, that can be more important than taking the entire application offline.

NIST recommends mechanisms that can supersede, disengage, or deactivate an AI system when its outcomes conflict with intended use. In production terms, you have several levels of containment. You can pause an execution queue, revoke a risky tool permission, switch to a safer fallback path, or require human approval before the next external action.

Choose the narrowest control that stops additional harm. If one tool integration is misbehaving, disable that authority rather than shutting down unrelated read only agent work. If the failure is in reasoning or policy selection, moving the whole workflow behind human approval may be the safer boundary.

Preserve evidence before making broad changes when that does not increase harm. Keep the trace, conversation identity, tool inputs and outputs, evaluation result, and latest workflow state. Those artifacts let you distinguish a model decision problem from a tool failure, a bad retry policy, or corrupted state.

This is also where the SentientOps incident response case study is relevant. The architectural lesson is not a particular vendor pattern. It is that incident controls need to exist before the incident, because improvising authority boundaries during an outage is slow and error prone.

Retries look like recovery until they repeat a side effect. That is why retry design belongs in the incident runbook rather than only in an SDK configuration.

Azure reliability guidance warns that poorly controlled retries can create cascading failures. It recommends finite retry counts, exponential backoff, randomization, and logging. Those controls reduce synchronized retry pressure and make the recovery behavior observable instead of invisible.

The deeper issue is whether the operation is safe to repeat. AWS describes this through idempotent APIs: a repeated request can return the original result without applying the mutation again. A unique request identifier gives the service a way to recognize that the caller is retrying the same intent rather than requesting a second action.

The AWS idempotency guidance matters especially for agents because tool calls often create external state. A repeated payment request can charge twice. A repeated notification can message a customer twice. A repeated workflow mutation can move the same record through a state transition twice.

Your recovery policy should therefore answer three questions before a failed tool call is retried. Is the failure plausibly transient? Is the operation safe to repeat? Can the service recognize the same intent if the request arrives again?

If any answer is no, stop automatic replay. Escalate to a human or use a compensating path that the system can reason about explicitly. A retry should reduce uncertainty, not multiply it.

The safest recovery point is usually the last known good state, not the beginning of the workflow. Replaying from the start can reissue actions that already succeeded before the incident became visible.

LangGraph persistence illustrates the state model clearly. Its persistence layer saves graph state as checkpoints and supports fault tolerance, time travel, and human review. The framework is only one implementation, but the underlying requirement is broader: durable agent workflows need enough state to resume intentionally.

Checkpointing changes the recovery question. Instead of asking whether you can run the agent again, ask which state is safe to resume from and which actions after that state must be reviewed before they execute.

Human control belongs at that boundary. NIST calls for clear human and AI roles and responsibilities, with intervention where risk warrants it. During an incident, that means the operator should have an explicit way to approve, reject, edit, or terminate the next action rather than relying on a prompt that merely asks the model to be careful.

Do not confuse a checkpoint with a guarantee. A saved state can contain a bad assumption, and an external side effect may already have happened. Recovery still needs reconciliation between stored workflow state and the systems the agent touched.

A simple sequence works well: identify the last reliable checkpoint, compare it with external state, mark completed side effects, require approval for ambiguous actions, then resume only the remaining work.

A green dashboard is necessary, but it is not enough. An agent can recover at the service layer while still returning degraded outcomes.

NIST's post deployment guidance treats monitoring, incident response, recovery, change management, and override mechanisms as part of the same operational lifecycle. That supports a stricter closure rule: the change is not complete until you have observed the repaired system under controlled traffic and checked the behavior that failed.

Start with operational verification. Confirm that error paths, latency, tool connectivity, retry volume, and queue behavior have returned to an expected shape. Then verify the agent outcome itself using the evaluation signal that originally exposed the incident, or a bounded evaluation set that represents the affected behavior.

OpenTelemetry's evaluation event is useful here because it places quality results beside normal telemetry. You can compare the repaired execution path with the failing one rather than treating model evaluation as a separate offline exercise.

Do not invent a universal pass threshold. The right threshold depends on the task and the risk of a wrong outcome. The runbook should define the evaluation set, owner, decision rule, and rollback path before the next incident.

Restore traffic gradually when you can. A bounded canary gives you a chance to see whether the fix holds under real requests while the containment controls are still easy to reapply.

The incident is not finished when traffic is stable. It is finished when the system is less likely to fail the same way without warning.

Google SRE treats significant undesirable events as postmortem candidates. Common triggers include user visible degradation, data loss, operator intervention, long resolution time, or a monitoring failure. The useful principle is not to write a document for every minor error. It is to document incidents that expose something the operating model needs to learn.

A strong postmortem records impact, timeline, detection gap, containment actions, recovery steps, and contributing conditions. Then it translates those findings into controls. If the team discovered the failure too late, add the missing trace or evaluation. If a retry duplicated an action, add idempotency. If an agent could reach a risky tool without review, narrow the permission or add an approval gate.

The postmortem should also update the incident runbook itself. New failure knowledge has little value if it stays in a document that nobody consults during the next outage.

For teams operating autonomous workflows, the agentic AI consulting service shows how observability, action controls, state, and human oversight fit into one production architecture. The point is not more process. It is a system that fails in ways your team can see, contain, and recover from.