← All articles
AI Agents3 min read

Your AI agent needs a runbook, not just a prompt

A prompt tells your agent what to do. A runbook tells your team what to do when it does something else. Write the failure modes, symptoms, and first three actions down before the 2 a.m. incident — not during it.

Your AI agent needs a runbook, not just a prompt

Every team ships its first agent with a beautiful prompt and no plan for the Tuesday it does something strange at 2 a.m. The prompt gets versioned, reviewed, and polished. The runbook — what to do when the agent refunds the wrong customer, loops on a tool call, or starts answering in a language nobody configured — doesn't exist. So the first incident becomes a group chat, a guess, and a hope.

A prompt tells the agent what to do. A runbook tells your team what to do when the agent does something else. That distinction is the whole job. The agent will fail in ways the prompt never imagined, because prompts describe the happy path and production is mostly the other paths. The runbook is where you write down the failure modes you already know about, the symptoms that identify each one, and the exact first three actions for each — who gets paged, what gets frozen, what gets rolled back.

Start with the five failures you can already name. The agent calling a tool twice. The agent quoting a policy that changed last quarter. The agent confidently completing a task it should have escalated. The agent slowing to a crawl because a retrieval step ballooned. The agent doing something nobody can explain afterwards. For each one, write the symptom a human would actually see, the check that confirms it, and the containment step that stops the bleeding before anyone debugs anything. Containment first, diagnosis second — the runbook exists so nobody has to be clever under pressure.

Then decide the kill switch in advance, while everyone is calm. What exactly freezes the agent — a feature flag, a revoked key, a paused queue — and who is allowed to pull it without asking permission? If the answer is "we'll decide in the moment," you don't have a kill switch, you have a meeting. The teams that recover fast are the ones where the newest hire can freeze the agent at 2 a.m. without waking anyone, because the runbook says so in plain language.

Keep the runbook where the incident happens, not in a wiki nobody opens. One page, linked from the same dashboard that shows the agent's health. Every incident adds a line to it: what happened, what the first responder did, what should have happened instead. A runbook that never changes is a runbook nobody uses.

We build AI systems your team can operate at 2 a.m. — prompts, runbooks, kill switches and the observability to know which one you need — because an agent you can't stop, explain, or recover is not a product. It's a liability with a nice demo.

Field note: A client once called us after their support agent started issuing refunds for orders that didn't exist — forty of them, over a weekend, before anyone noticed. The prompt was fine. The retrieval step was returning the wrong order records, and the agent trusted them. There was no runbook, so the first responder spent an hour in the codebase while the agent kept refunding. We wrote the runbook in an afternoon: the symptom to watch for, the flag that freezes refunds, the person who can pull it without approval. The next incident took four minutes to contain. The prompt barely changed. The operations around it did.

Related service Workflow automation

Building something worth shipping?

We take on a small number of AI product engagements. Tell us what you are building — we reply quickly.