Server dashboards can glow green while viewers stare at spinners: edge metrics measure what you sent, not what anyone experienced. Quality of experience is defined at the player or it is fiction — and once measured there, the discipline is choosing few enough numbers to act on, cutting them by the dimensions that assign blame, and wiring alerts to the cuts, because that is where trouble starts.
QoE is the player’s view, or it is fiction
Every number in this guide comes from the player beacon: events for session start, first frame, stalls, rung switches, errors and heartbeats, shipped to your RUM pipeline with session context attached. The instrumentation is the video twin of deploying RUM — sampling policy, beacon reliability on page close, dimension capture — with one video-specific ruling to make on day one: define a “view” and a “valid session” precisely (does a two-second bounce count? a preview autoplay?), because every ratio downstream divides by it, and an undefined denominator is how two dashboards show two truths.
The core four, and how to define them
Four metrics carry the experience; resist the urge to track twenty. Startup time: first-frame latency from play intent, reported at percentiles (p50 for typical, p95 for the experience that loses people) — the levers live in the startup checklist. Stalls: rebuffer ratio for severity plus stall frequency for annoyance, as unpacked in fighting rebuffering. Delivered quality: audience-weighted delivered bitrate or time-on-top-rungs — the counterweight metric, because stalls can always be bought with ugliness, and only a quality metric exposes the purchase. Failure rate: sessions that never render a frame (exit-before-start) plus fatal in-play errors — small percentages, total losses, and the number most estates discover late. Track the four together on one view; each is gameable alone, and their tension is the point.
Dimensions: the cuts that assign blame
A global average is a press release; the cuts are the diagnosis. Capture per session: CDN (indispensable the moment you run two), geography and ASN, device class and player version, connection type, live vs VOD, title, and starting rung. These are the axes along which causes separate — one ASN degrading across CDNs is a network path; one CDN degrading across ASNs is delivery; one player version degrading everywhere is a regression; one title degrading is encoding — the same truth-table logic as rebuffer localization, now applied to all four metrics. Cardinality is the practical constraint: pick the handful of dimensions you will actually cut by and enforce them in the beacon schema, because a dimension not captured at session time cannot be reconstructed at incident time.
Thresholds and alerts that page correctly
Alert design decides whether QoE is a control system or wallpaper. Alert on the cuts, not the blend: a per-ASN, per-CDN stall regression pages while the global average sleeps — which is correct, because incidents start local. Use baselines rather than fixed absolutes where traffic is cyclic (weekend-evening stall ratios differ from Tuesday mornings), but keep hard floors on the catastrophic signals: exit-before-start ratio and fatal error rate get absolute thresholds and page immediately, since they spike within a minute of a broken token config, an expired certificate or a dead license server. Route by severity — failure spikes page, quality-mix drifts open tickets — and hold the line on alert honesty with the same discipline as synthetic monitoring: every alert either drove an action or gets retuned. Pair the RUM alerts with a thin synthetic layer (scripted playbacks from fixed vantage points) so you still see at 4 a.m. when real-viewer volume is too thin to alarm on.
Closing the loop: QoE as a control signal
Measured and alarmed, QoE graduates from reporting to control. Release guardrail: the core four, cut by the standard dimensions, compared automatically across every player build, ladder change and delivery config push — regressions block, which converts slow churn-side discoveries into same-day rollbacks. Steering signal: in multi-CDN estates, per-ASN-per-CDN QoE is the honest routing input — shift traffic toward whichever platform delivers better into each network, and you have closed the measurement loop. Commercial evidence: provider conversations change tone when you arrive with per-region QoE by platform rather than anecdotes. And planning input: the delivered-quality mix feeds the capacity arithmetic for the next big night in live-event delivery. The estates with the best viewer experience are not the ones with the most metrics — they are the ones where four numbers, cut correctly, are allowed to stop a release and move the traffic.