Every major delivery network now sells AI inference, and the marketing has converged on a single phrase: run the model close to the user. The claim is true and the arithmetic is more specific than the phrasing suggests. GPU capacity does not live at every cache node and never will, which means “the edge” in edge inference is a different, much smaller map than the one on the coverage page.
The GPUs are in tens of cities, not thousands of PoPs.
Delivery networks have thousands of points of presence and inference in a few dozen of them, because GPUs need power, cooling and utilisation that cache nodes do not have. That tiering is sensible engineering, and it means the buying question is not how big the network is but where the inference locations are relative to your users, and whether your workload is latency-bound in the first place.
Why the map is smaller than the network
A caching node is cheap, small and useful at low utilisation: it fills a rack, draws modest power, and earns its keep serving bytes. An inference node is none of those things. Current server-class accelerators are power-hungry and need cooling and electrical capacity that most edge sites simply do not have, and they only make financial sense at high utilisation — an idle GPU is one of the most expensive idle things in infrastructure.
The result is a two-tier architecture across every vendor selling this. Delivery and lightweight compute run everywhere; GPU inference runs in a selected subset of well-provisioned facilities. Akamai has been unusually candid about the economics, deploying its inference platform into roughly twenty locations with a stated path to perhaps forty, while explicitly noting that GPUs across thousands of points of presence would not be economic. Cloudflare has taken the opposite approach at the density end, putting inference capacity into a large number of cities and leaning on that breadth as the differentiator. Gcore built its offering around GPU inference from a smaller but deliberately positioned footprint.
None of that is a criticism. It is the correct shape for the physics. But it changes the question a buyer should ask from “how many PoPs do you have” to “which of your sites can run my model, and how far are they from my users?”
Which workloads the geography actually helps
Network latency is a fixed cost you can move; generation time is not. If a request spends twenty milliseconds crossing the network and five milliseconds in the model, moving the model closer is transformative. If it spends twenty milliseconds crossing the network and four seconds generating tokens, the same move is a rounding error. That ratio is the entire test, and it sorts workloads cleanly.
The clear wins are small and fast: classification, embedding lookups, semantic search over a cached vector index, language and intent detection, content moderation, fraud and risk scoring, personalisation decisions, and routing choices that determine which larger model to invoke. These are single-digit to low-double-digit millisecond operations where network time dominates, and they are frequently in the critical path of a page render, which multiplies the benefit.
The weak cases are long-form generation, anything requiring a frontier-scale model, and anything whose inputs live in one database in one region. In that last case moving the model to the edge simply relocates the round trip, because the model still has to reach the data. The rule that survives contact with production: put the compute where the latency-sensitive decision is made, and keep it away from where the state lives unless you have moved the state too.
The economics that make it interesting
The commercial argument is often stronger than the latency one, and it is about egress. Running inference in a hyperscaler region and serving the results out through a CDN means paying data transfer at hyperscaler rates on every response. Providers running inference on their own delivery network can charge nothing between compute and edge, because there is no third party in between. For a chatty inference API returning modest payloads at high volume, that structural difference can outweigh the per-token comparison entirely — the same dynamic we examined in egress fees.
The second economic factor is availability rather than price. GPU capacity has been genuinely scarce, and providers have reported selling out allocations and passing on price increases — a market condition, not a policy. Anyone planning capacity for a launch should be asking about lead times and reservation terms with the same seriousness they apply to bandwidth commitments, and should read any take-or-pay clause carefully before signing.
Third, watch what the bundling does to negotiation. When delivery, security and inference sit on one contract, the discount conversation becomes multi-dimensional, which cuts both ways: more levers to pull, and more of your estate tied to one renewal. That is a familiar trade, covered in the renewal playbook.
What to ask a vendor
Five questions separate a real evaluation from a demo. Which specific locations run inference today, and which are roadmap? What is measured latency from those locations to my top five audience regions, at the ninety-fifth percentile rather than the mean? Which models can I run — a fixed catalogue, or my own weights — and what happens when I want a model the catalogue does not carry? How is it billed: per token, per second of compute, per request, and what does the egress line look like? And what is the isolation story, given that my inputs may be regulated data passing through a shared platform?
That last one is doing more work every quarter. Inference is data processing, and where it happens is a jurisdictional question, not only a performance one. A model running in an edge location in another legal territory is a data transfer, whatever the marketing calls it — which is why sovereignty and inference are converging into the same procurement conversation, as covered in why sovereignty became a delivery problem.
The realistic architecture
What works in production is rarely all-edge or all-origin. It is a router: a small, fast model at the edge classifies the request and handles the easy majority, escalating the minority that genuinely needs a larger model to a regional GPU cluster. Most requests are answered in tens of milliseconds; the hard ones cost a round trip and are still inside budget. Caching helps more than teams expect — identical or near-identical prompts recur far more often than intuition suggests, and a response cache in front of inference is the cheapest optimisation available.
Treat model deployment like any other edge configuration: versioned, staged, observable, reversible. A model that regressed silently across a hundred locations is a worse incident than a bad cache rule, because the failure is statistical rather than binary and your monitoring probably is not looking for it. The deployment discipline in canary deployment at the edge applies directly.
The honest summary for 2026: edge inference is real, the geography is narrower than the brochures imply, and the workloads it transforms are a specific and identifiable minority. Buy it for those, price it on egress as well as compute, and ignore the PoP count entirely — it is measuring the wrong network.