Every major CDN has had a bad day, and when one does, thousands of teams discover simultaneously that their site’s availability was an assumption. The incident itself is not the interesting part — you cannot fix their infrastructure — the interesting part is the ninety minutes on your side: knowing it is them, knowing your options, and executing one you rehearsed instead of improvising one you did not. This playbook is written before the incident; during it, you only read.
Detection you do not rent from the patient
You cannot detect a CDN outage through the CDN: its dashboard, logs and status page are all served by, or updated at the discretion of, the party having the bad day — and status pages historically lag real impact by many minutes. Run external synthetic probes against your production hostnames from several regions, through an independent provider, alongside one probe against your origin directly (or a lightweight bypass hostname) so you can immediately separate “edge is failing” from “everything is failing.” Alert on the differential: edge probes red while the origin probe is green is the signature this playbook exists for. The setup details are in synthetic monitoring without false alarms; the incident-day payoff is a diagnosis in two minutes instead of twenty.
Diagnosis: CDN, DNS, origin or you?
The first fork in the tree: what exactly is broken? Widespread 5xx from edge nodes with a healthy origin points at the CDN’s serving layer. Resolution failures point at DNS — which may be the same vendor or a different one, and changes your options entirely. Edge errors naming the origin (bad-gateway responses) may mean the CDN is fine and your origin, or the path to it, is the patient — check before blaming, and see the triage flowchart for the request-level walk. And always ask what changed on your side in the last hour: a config push at the same moment as an outage is either coincidence or cause, and rolling back your own change is the cheapest test available. Confirm with outside evidence — independent outage trackers and the provider’s incident feed — because the response to “global CDN incident” and “something wrong with our account” are different: the second one starts with a support ticket, not a failover.
The options ladder, priced in advance
Write down your realistic options and their true costs on a quiet day, because each has prerequisites that cannot be conjured mid-incident. Option zero: ride it out — correct more often than pride admits, when the outage is partial or short; your traffic profile decides. Option one: degrade gracefully — if your app can serve a lighter static experience or queue writes, edge failure hurts less; this is an application capability, not a CDN setting. Option two: bypass to origin — point DNS at the origin directly; prerequisites are short DNS TTLs already in place, origin capacity honestly assessed for a cache-less load spike (see origin-load reduction), and a decision about the origin protections that bypass will violate — an origin locked to CDN ranges or reachable only privately cannot fail open, a trade-off chosen deliberately in origin protection. Option three: fail over to a second CDN — the strongest answer and the most demanding, requiring config parity, certificates already issued, and DNS or steering rehearsed; the full setup is its own guide. Whichever subset applies to you, the playbook states the trigger conditions: how many minutes, what error rate, which business windows justify which rung.
During: execution and communication
With detection ringing and diagnosis pointing at the CDN, the incident becomes execution: one person drives the technical ladder, one owns communication, and nobody freelances new ideas mid-storm. Communication runs on templates written in advance — internal notice, status-page entry (hosted off the affected infrastructure, or it will be down too), support macro — each stating impact, action, and the time of the next update. Meanwhile, capture evidence continuously: probe results with timestamps, error samples, the provider’s incident timeline, your own action log. It feels bureaucratic at minute twenty; it is decisive at the SLA claim and invaluable at the postmortem. If you executed a bypass or failover, decide the return criteria now, not later: providers recover unevenly, and flapping back onto a half-healed edge is a second incident.
After: recovery, credits, postmortem
Return deliberately — canary a low-stakes hostname onto the recovered CDN, watch error rates, then move the rest, and restore any protections the bypass loosened (re-lock the origin, revert TTLs, rotate anything exposed). File the SLA claim while evidence is fresh: credits are typically claimed, not automatic, with deadlines and required documentation in the contract — your timestamped probe data is exactly what substantiates impact beyond the provider’s own accounting. Then hold the postmortem on your response, not their outage: how long to detect, how long to diagnose, which ladder rung you took and whether its prerequisites actually held. The honest output is usually a short list — TTLs to shorten, a rehearsal to schedule, an origin capacity number to fix — and sometimes the bigger conclusion that your availability requirements have outgrown a single vendor, which is a procurement decision to make calmly, priced against what the incident actually cost.