What GPU Compute Actually Costs

When a team compares Aegis Private AI to what they pay OpenAI or Anthropic today, the instinct is to line up total monthly spend against total monthly spend. That comparison is wrong, and it flatters the API. Frontier APIs and dedicated infrastructure price different things. The API price bundles the compute, the platform engineering, the MLOps, and the vendor's margin into a single per-token number. Dedicated infrastructure prices the compute, and the operational work sits in a separate managed services line.

To compare honestly, strip both sides back to the compute layer. The tables on this page do exactly that: raw GPU infrastructure cost against raw API token cost. On the Aegis side, managed services are additional. On the API side, that same operational work exists too, it is just invisible in the price because you are the one doing it, or going without it.

There are three cost models in the market, and most enterprises only have visibility into the first.

Per-token API pricing. You pay per million tokens and never see the infrastructure. Fast to start, impossible to forecast at scale, and the model under which your true consumption stays hidden.

Hyperscaler GPU instances. AWS EC2 (G5, P4, P5), SageMaker, and the equivalents on Azure and Google. You rent named GPU capacity with enterprise SLAs, IAM, and compliance attestations built in. Convenient and governed, but you pay a premium: hyperscaler A100 and H100 rates typically run 40 to 65 percent above specialty providers.

Specialty GPU clouds. Lambda, CoreWeave, RunPod, Vast.ai. Purpose-built for GPU workloads at materially lower rates. The trade is fewer enterprise governance features and, on marketplace-style providers, interruption risk. This is where the economics lead when governance does not force the hyperscaler.

All figures on this page are directional and dated. GPU pricing shifts monthly.

The 40 to 65 percent premium is not waste. It buys enterprise SLAs, native IAM integration, compliance attestations, and tight ecosystem integration with the rest of your cloud estate. If your governance model requires AWS-native controls, or the workload lives inside an existing AWS security boundary, that premium is the price of not re-architecting your compliance posture. The mistake is paying it by default, for workloads that never needed it.

Both models carry costs that do not appear on the headline price. On the dedicated side: storage, egress, idle capacity during ramp, engineering time, and on marketplace providers, interruption risk. On the API side: the same egress and storage, plus the entirely hidden cost of never being able to size, forecast, or own your infrastructure. The honest comparison accounts for both. The dishonest one compares a fully-loaded on-prem estimate against a headline API price and declares the API cheaper.

The intuition that your own GPUs cost more than APIs is only true during pilots and experimentation, when utilization is low. At production utilization the arithmetic reverses, and it reverses hard.

Break-even is the monthly token volume at which dedicated infrastructure cost equals API cost. Above it, dedicated is cheaper per token. Below it, APIs are. For an 8-GPU Foundation tier against the Sonnet 5 tier, break-even lands around 3.2 billion tokens per month, roughly 24 percent of the cluster's peak capacity. Most production workloads clear that within months. Move the sliders below to find your own crossover.

There is no single right provider, only the right provider for a given workload and governance model. IVI selects AWS-native infrastructure when governance demands it, and specialty providers such as Digital Ocean when the economics lead and the controls are satisfied another way. Because the managed services fee is priced against the alternative of running your own ML platform team, not against the GPU cost, the underlying compute stays a transparent, low-margin pass-through, disclosed at 10 to 15 percent. You get the provider that fits, not the provider that pads the margin.

Private AI isn't the right call for every team, and we'd rather tell you that in the first conversation than the sixth month. Here's how we think about fit.