Emergency medicine trains people to coordinate under pressure through defined roles, standardized communication, repeated simulations, debriefs, practical exams, and recertification. DevOps often teaches technical tools, gives someone an outdated runbook, and then puts them on call without meaningful practice in leading a real incident.
I keep wondering what a true production-incident equivalent of ACLS would look like. Imagine an intensive, vendor-neutral course where a group of engineers operates a deliberately messy fictional company with APIs, databases, queues, monitoring, deployments, source control, chat, ticketing, broken documentation, confusing access controls, and misleading system names. Without warning, production incidents occur several times throughout the week.
The participants would rotate through roles such as Incident Commander, technical lead, communications, scribe, and responder. After each exercise, they would debrief, identify process and human-factor failures, and improve the fictional company's runbooks, access procedures, monitoring, escalation paths, and incident practices. Later scenarios would introduce misleading telemetry, cascading failures, executive interruptions, conflicting diagnoses, and unsafe recovery attempts.
The final assessment would put the group into an unfamiliar outage and evaluate how well they establish command, define customer impact, coordinate work, communicate clearly, test hypotheses, and restore service safely—not merely whether one person can find the faulty line of code.
There are already GameDays, chaos exercises, incident-management frameworks, simulators, and certifications. But is there a cohesive system that combines incident doctrine, human factors, repeated live simulation, rotating roles, practical assessment, and ongoing recertification? If not, what is stopping the industry from creating one?
3 Answers
The biggest challenge is that every organization has different systems, ownership boundaries, escalation rules, and cultural problems. A standardized course can teach general principles—command structure, role clarity, communication, decision-making, and debriefing—but it cannot perfectly reproduce the organizational quirks that make real incidents difficult. Companies would probably need to combine a common core with scenarios tailored to their own environment.
The cost is probably another major barrier. Taking eight engineers away from their normal work for a week is already expensive, before paying for instructors, a realistic disposable environment, scenario authors, observability, and evaluation. Many organizations only invest after a serious outage, even though repeated simulation would likely be much cheaper than learning those lessons during a customer-facing disaster.
The closest equivalents exist, but they are usually separate products or internal programs rather than one universal certification. GameDays, incident-command training, chaos exercises, cyber ranges, and incident-management platforms each cover part of the problem. A complete course would require a lot of scenario design, realistic infrastructure, facilitation, assessment, and constant maintenance, so it would be expensive to build and deliver.
That makes sense, especially because the value is in the realism and repetition rather than in simply memorizing a framework.