Raw view: Runbook From an Incident
No invisible, zero-width, or bidi-control characters detected.
Write a runbook for {{SYSTEM_NAME}} based on the incident below. The reader is an on-call engineer at 3 a.m. who has never seen this failure, so every step must be an action they can execute, with the exact command or click, and the observation that tells them what to do next.
<postmortem>
{{POSTMORTEM}}
</postmortem>
<incident_chat>
{{INCIDENT_CHAT}}
</incident_chat>
<existing_runbooks>
{{EXISTING_RUNBOOKS}}
</existing_runbooks>
<tooling>
{{TOOLING}}
</tooling>
<incident_chat> is where the real commands live; the postmortem usually summarizes them away. <existing_runbooks> shows the house format and what already exists, so you extend rather than duplicate. <tooling> names the dashboards, CLIs, and access paths available (for example, "kubectl via bastion, Grafana folder 'payments', PagerDuty service X").
Rules:
1. Mine <incident_chat> for every command that was run, every dashboard that was checked, and every observation that changed the responders' minds. Quote commands exactly; they were run against the real system and worked. Where the chat shows a command that was tried and did not help, include it under "Known dead ends" so the next responder skips it.
2. Structure the runbook as a decision tree, not a story. Each step: the action, the expected observation if the hypothesis holds, and where to go if it does not. A step with no branch is a step the reader will not know how to interpret.
3. Lead with the symptom as the alert or user report would present it, and the fastest confirmation that this is the same failure mode and not a lookalike. False positives waste the first ten minutes.
4. Separate mitigation (stop the bleeding) from remediation (fix the cause). The mitigation that worked in the incident goes first, with its blast radius and how to roll it back; remediation is a pointer to the ticket or fix, not a procedure.
5. Flag anything that requires elevated access, affects customers, or is irreversible with a visible `CAUTION:` line immediately above the step, stating what it affects.
6. Where the postmortem names an action item that would make this runbook obsolete (an auto-remediation, a fix), note it at the top so the runbook can be retired when it lands.
Replace any internal hostnames, account IDs, or secrets that appear in the chat with placeholders like `<prod-db-host>` and list them under "Placeholders to fill". Use the vocabulary and heading style of <existing_runbooks> if provided.
## Output format
```
# Runbook: <failure mode name>
**System:** {{SYSTEM_NAME}} | **Severity when seen:** | **Last incident:** <date> | **Owner:**
**Retire when:** <action item that makes this obsolete, or "n/a">
## Symptoms (how it presents: alert names, error strings, user-visible effect)
## Confirm it is this (1-3 checks with exact commands/dashboards and expected output)
## Immediate mitigation (numbered; each step: action, expected result, rollback; CAUTION lines where needed)
## Diagnosis (decision tree: step, observation, next step; exact commands)
## Known dead ends (what was tried in the incident and did not help, and why)
## Escalation (who, when, with what information)
## Remediation (pointer to the fix/ticket; not a procedure)
## Related (links to <existing_runbooks> entries and dashboards)
```
After the runbook: **Placeholders to fill** (each `<placeholder>` and what it stands for) and **Gaps** (steps you could not make concrete from the inputs, with what information would close them).