Synthetic tests and real-user monitoring answer different questions. CDN decisions keep going wrong because buyers ask the synthetic question and act on it as if it were the real one.
What synthetic is for
Controlled probes from fixed locations are excellent at regression-catching: same test, same place, did the number move after the deploy? They are consistent precisely because they are unrealistic, clean machines on clean networks in data centers where no customers live. The confusion persists because both produce charts with the same units, and charts with the same units feel interchangeable right up until a decision depends on them.
What RUM knows that labs cannot
Your actual users on actual networks: the phone on a train, the office behind a slow proxy, the whole messy distribution. RUM reveals the p95 tail where CDN differences genuinely live, weighted by who actually visits, not by where probes were rented. The weighting property is the underrated half: RUM automatically samples your actual audience mix, so growth in a new market shows up in the data before anyone remembers to add a probe there.
Sample-size honesty matters as RUM adoption grows: percentiles from thin cohorts mislead confidently. A region with a few hundred sessions produces a p95 that jumps around like weather, and decisions built on it inherit the noise. The discipline is stating minimum cohort sizes before reading regional splits, and letting thin regions accumulate before judging them. Synthetic probes, for all their unreality, never have this problem, which is the honest reason to keep a few running: they are the stable baseline against which RUM’s rich, noisy truth gets sanity-checked.
The decision rule
Choose and judge providers on field data; use synthetic for monitoring what you chose. Any provider comparison built on lab scores alone deserves the question: measured on whom? The speed receipt across this site is field data by construction, your browser, this visit. The decision rule has a corollary for incumbents: judge your current provider on field data too, because a network chosen on lab scores years ago may be quietly failing a tail it was never measured on.
In practice
Stand up basic RUM this quarter if you have none: the browser timing APIs are free, the collection endpoints are trivial, and the first month of field percentiles will recalibrate more opinions than any vendor meeting. Keep one synthetic probe per critical region for regression alerts. With both in place, every future CDN conversation starts from your users’ reality instead of a rented machine’s.
Our benchmarks are RUM-based for this reason. The assessment shows the distribution, not a lobby-friendly average.