← All articles
Production AI4 min read

Who gets paged when the agent misbehaves

Your AI agent will misbehave at 2 a.m. — and it fails politely, in fluent sentences. On-call for AI means monitoring behaviour, not just uptime: a real kill switch, an owner with authority, and drills before the incident.

Who gets paged when the agent misbehaves

Your AI agent will misbehave at 2 a.m. The question is not whether it happens — it is who gets the page, what they can actually do about it, and how long the bad behaviour runs before anyone notices.

Most teams ship agents with no on-call plan at all. The model is in production, the dashboard shows green, and the first sign of trouble is a customer email the next morning. By then the agent has spent hours confidently doing the wrong thing.

On-call for AI is different from on-call for a server. A server fails loudly. An agent fails politely, in fluent sentences, with full confidence. Your alerting has to catch behaviour, not just errors.

Monitor behaviour, not just uptime

Latency and error rates still matter, but they miss the failures that cost real money. Track what the agent is doing: escalation rate, tool-call failures, the share of conversations ending in a human handoff, and — most telling — the rate at which it takes irreversible actions.

A refund rate that doubles overnight is an incident even if every API call returned 200. A sudden drop in escalations is not a win; it usually means the agent stopped asking for help and started guessing.

The page needs an owner with real power

Somebody has to own the agent the way an SRE owns a service. That person needs a kill switch that actually stops the agent, a way to roll back to the previous prompt or model version in minutes, and the authority to use both without a meeting.

The kill switch is non-negotiable. If the only way to stop a misbehaving agent is to deploy a code change, you do not have a kill switch — you have a hope. A feature flag, a traffic dial turned to zero, a circuit breaker on the tool it is misusing: pick one and test it before you need it.

Run the drill before the incident

Schedule a game day: deliberately degrade the agent — swap in a stale knowledge base, cut off a tool, inject a prompt that triggers a wrong refund — and watch how long it takes the team to notice, page the right person, and stop it. Most teams discover their alerting fires on the wrong things and their runbook says "contact the vendor," which is not a runbook.

Write the runbook from what the drill teaches: the symptoms that mean "freeze the agent now," the exact steps to do it, and who decides to unfreeze it. An agent that cannot be stopped safely cannot be trusted in production, no matter how good the demo was.

We help teams put AI systems into production with the guardrails that make them operable — kill switches, behaviour monitoring, and an on-call plan — because a system nobody can stop at 2 a.m. is not a system. It is a liability with a login.

Field note: In one rollout, an agent started issuing duplicate refunds after a model update changed how it parsed a confirmation field. Nothing errored. The first alert came from a finance analyst three days later. The fix took an hour; the recovery took a week. The team now has a refund-rate threshold that pages a human in minutes, and a kill switch they have actually tested.

Related service AI products

Building something worth shipping?

We take on a small number of AI product engagements. Tell us what you are building — we reply quickly.