The Runbook Problem: Why Most Runbooks Are Already Out of Date
AI SRE Runbook Automation — A Different Approach
Here is the argument this post is making: the runbook problem is not that engineers don’t write runbooks. They do. The problem is that runbooks are documentation, and documentation degrades. Every deployment that changes a service, every dependency that shifts, every threshold that gets recalibrated — each one creates a small gap between what the runbook says and what the system actually does. Those gaps accumulate silently. The runbook looks complete. The engineer following it at 2am trusts it. Then it doesn’t work.
AI SRE runbook automation doesn’t solve this problem by making runbooks easier to write or better organised. It solves it by making the runbook layer executable, observable, and self-correcting — owned by Coworker rather than maintained by engineers who have other things to do.
This is a different claim from “automation is faster than manual execution.” That is true but obvious. The deeper claim is that the runbook problem is fundamentally a maintenance problem, and maintenance problems are solved by removing the human from the maintenance loop, not by improving the tools the human uses to maintain.
Why Runbooks Go Stale
A runbook is written at a specific moment in the history of a system. It reflects the service topology, the dependency map, the threshold values, and the remediation steps that were correct when it was written. The moment any of those things change — and in a living production system, something changes every week — the runbook begins to drift.
The drift is usually small at first. A service was renamed. A connection pool threshold was adjusted. A new dependency was added that the runbook doesn’t account for. The runbook still works for 80% of cases, which is why nobody updates it. The 20% of cases where it doesn’t work are discovered during incidents, under pressure, by engineers who trusted the runbook enough to follow it before questioning it.
The incentive structure makes this worse. Writing a runbook is visible, credited work — it produces a document that can be pointed to. Updating a runbook is invisible, uncredited work — it produces a revision to a document that nobody reads unless something is going wrong. The result is that runbooks accumulate updates in their drafts and never get them in their published versions.
There are teams that invest in runbook maintenance. They schedule quarterly reviews. They assign ownership to specific engineers. They use tools that surface runbooks that haven’t been reviewed recently. These teams have better runbooks than teams that don’t do this. They still have stale runbooks — because the review cadence is always slower than the rate of system change. A quarterly review finds the runbooks that drifted in the last quarter. It doesn’t catch the runbook that drifted last Tuesday when the deployment team changed the service configuration.
The fundamental problem is not process. It is that runbooks are documents, and documents can’t observe the system they describe.
What Changes When Coworker Owns the Runbook Layer
Coworker’s approach to ai sre runbook automation is not to make runbooks better documents. It is to make runbooks executable artifacts that are derived from observed system behaviour rather than written from memory.
When Coworker identifies a recurring pattern — connection pool pressure on the payment service that has occurred four times in the last 90 days, resolved each time by reducing batch concurrency — it doesn’t record this as a text document. It records it as a structured pattern-action pair: the specific signal combination that indicates the pattern, the specific action that resolved it, the services affected, the resolution time, the confidence level based on how consistent the resolution has been.
That pattern-action pair is the runbook. It was not written by an engineer. It was derived from observed incidents. It reflects the system as it actually is, not as it was when a document was last edited.
When the system changes — when the batch job is refactored and the signal combination that indicated the pattern no longer matches the same signals — Coworker’s pattern detection adapts. The old pattern may no longer fire. A new pattern may emerge. The runbook is not a document that needs to be updated. It is a model that updates itself as the system changes.
This is what makes ai sre runbook automation categorically different from traditional runbook automation tools. Traditional runbook automation takes a document and makes it executable — it runs the steps that a human wrote. Coworker derives the runbook from observed behaviour and keeps it current by observing the system continuously. The human writes nothing. The runbook can’t go stale because it was never static.
As we covered in AI SRE Incident Memory, this is exactly how incident memory works: every situation Coworker surfaces and every resolution it observes becomes part of its operational knowledge of the specific stack. The runbook is a product of that knowledge, not an input to it.
The Approval Step Matters
There is a reasonable objection to this argument: if Coworker derives runbooks from observed behaviour without human authorship, how does the team know the runbook is correct?
The answer is the approval step. As we covered in Self-Healing Runbooks, Coworker’s autonomous runbook execution requires explicit team approval for each pattern. Coworker identifies the runbook candidate and presents it to the team for review. The team decides whether the proposed action is correct, whether the confidence level is sufficient, and whether the service criticality justifies autonomous execution.
The approval step does not require the team to write or maintain the runbook. It requires the team to review and authorize a runbook that Coworker has derived from evidence. That is a materially lower maintenance burden than writing and maintaining runbooks from scratch — and it comes with an audit trail of how the runbook was derived, which observed incidents it is based on, and what the confidence level is at the time of approval.
The team remains in control. The runbook remains human-authorized. But the maintenance work — the continuous observation, the pattern recognition, the confidence calibration, the gap detection when the system changes — is done by Coworker, not by engineers.
The Honest Limitation
The argument in this post has one important limitation worth naming: Coworker’s runbook automation works best for recurring, recognizable patterns. For novel failure modes — the incident that has never happened before in a specific form — there is no pattern-action pair to derive, no runbook candidate to surface, no autonomous execution to approve. Novel incidents require human investigation and response.
This is not a flaw in the approach. Novel incidents should require human judgment. The goal of ai sre runbook automation is not to automate every incident — it is to automate the recurring, recognizable failures that represent the bulk of incident volume and the bulk of the on-call burden. Freeing engineers from the 70% of incidents that are variations of known patterns leaves them more capacity — and less burnout — for the 30% that require genuine investigation.
For how the shift from reactive to proactive operations changes the incident mix over time, see Proactive SRE. For the full incident management picture, see AI SRE Platform and Incident Management.
The Coworker page covers the full capability set. For pricing — no form, no sales call.
Frequently Asked Questions
They remain unchanged and valid. Coworker does not replace or deprecate existing runbooks — your Confluence pages, PagerDuty runbooks, and any other documentation stay in place. Coworker builds its own pattern-action knowledge alongside your existing documentation through observed incidents. Over time, as Coworker accumulates incident memory for your specific patterns, its derived runbooks become the primary response mechanism for recurring patterns, and the existing documentation serves as a reference layer for novel incidents.
Coworker's runbook derivation is based on observed system behaviour rather than imported documentation — importing a document doesn't give Coworker the observational evidence it needs to calibrate confidence and derive pattern-action pairs. However, existing runbook documentation is valuable context during the approval step: if a team reviews a Coworker runbook candidate and recognises that the proposed action matches what their existing documentation recommends, that alignment increases confidence in the approval.
Low-frequency incidents produce fewer observations, which means Coworker accumulates pattern-action pairs more slowly. A pattern that occurs once every six months will take longer to reach the confidence threshold for runbook candidate status than a pattern that occurs every fortnight. For very low-frequency patterns, Coworker surfaces the historical context when the pattern recurs — even if the pattern hasn't met the threshold for autonomous execution approval. The historical context is valuable even before autonomous execution is appropriate.
Coworker can execute runbook actions across both application and infrastructure layers, within the scope of the pre-authorized action library configured during onboarding. Infrastructure-level actions — scaling resources, restarting pods, adjusting Kubernetes configurations — are supported but typically require more conservative approval thresholds than application-level actions, given the higher blast radius of incorrect execution. The team controls the action scope and approval thresholds at onboarding.
Stop maintaining runbooks. Let Coworker derive them from what actually happens.
Start your free trial → app.opspilot.com/sign-up
Or talk it through: Book a demo → calendly.com/fusionreactor-sales/opspilot-demo
OpsPilot is the AI SRE teammate for teams using OpenTelemetry, Prometheus, Grafana, and existing observability stacks — helping engineers investigate incidents, find root cause, and move toward autonomous operations without replacing their tools. OpsPilot, formerly FusionReactor Cloud, is Intergral’s AI-powered observability and AI SRE platform.