The hourly rate on a cloud GPU pricing page describes almost nothing about what a production workload actually costs. It leaves out egress, the storage tier your checkpoints have quietly accumulated in, what happens to the bill when a run overruns its estimate, and a handful of other charges that only become visible once someone sits down with twelve months of invoices instead of one. That review rarely happens on a schedule. It happens when finance asks why the infrastructure line grew faster than output did.

The questions below are the ones worth running against your own billing data this week, not against a vendor's claims.

1. What is your GPU utilization over the last 30 days, measured against what you paid for?

Cast AI's 2026 State of Kubernetes Optimization Report, drawn from tens of thousands of clusters, put average GPU utilization at 5%. At that level, the effective cost per utilized GPU-hour on an on-demand H100 instance works out to roughly 20 times the nominal rate, since the vast majority of provisioned capacity sits idle. Pull your own utilization metrics from your monitoring stack, not your billing dashboard, and compare them to the compute hours on the invoice.

2. What did egress cost you last quarter, broken out as its own line?

Public egress on AWS, Azure, and GCP runs $0.05 to $0.12 per gigabyte depending on volume tier. Industry benchmarks put egress at 6% to 12% of a typical cloud bill, but for data-intensive AI workloads, teams moving large training sets or model weights regularly, that share can climb to 20 to 30% of total infrastructure cost. Most teams don't isolate it as its own line, which is exactly why it's worth pulling separately.

3. Does your contract cap the cost of a training run that exceeds its estimate?

On usage-based billing, a run that takes 40% longer than planned costs 40% more, with no ceiling unless it's been specifically negotiated. This matters more for long training and fine-tuning jobs than short inference workloads, since the variance compounds over days or weeks. Ask whether an overrun triggers an alert before it triggers a charge.

4. How much is checkpoint and dataset sprawl actually costing you?

A single 70B-parameter model saving hourly checkpoints generates roughly 6.7TB of storage per day. Multiply that across every experiment, retrain, and abandoned branch, and the volume compounds fast, especially since cloud providers typically charge four to six times more to retrieve data than to store it. Old checkpoints and superseded dataset versions rarely have an owner assigned to clean them up, which is exactly why they're usually the easiest line item to cut without touching production.

5. What's your effective rate after reservations, spot pricing, and negotiated savings?

List price and effective price diverge fast. Spot and preemptible instances on the major clouds are discounted steeply against on-demand rates, but they come with interruption risk that doesn't show up in the savings headline. If your team can't produce the blended effective rate without pulling in finance, that's a sign the true cost isn't visible to the people making architecture decisions.

6. Is support bundled into your rate, or billed as a percentage of spend?

Several major providers price premium support tiers as a percentage of monthly usage, which means the cost of getting a fast response scales with how much you're already paying them, independent of how much support you actually need. Confirm what tier you're on and whether its response-time SLA has ever been tested against a real incident.

7. What are your non-production environments costing while idle?

Cast AI's 2026 report also found CPU overprovisioning jumped from 40% to 69% year over year across Kubernetes clusters, with memory overprovisioning sitting at 79%. Dev, staging, and experimentation environments are usually the biggest contributor, provisioned once for a project and left running long after nobody owns the decommissioning step. A utilization audit on non-prod instances is typically the fastest way to find savings with zero architectural change.

8. How much does your total GPU spend vary month over month, and can you explain the variance?

Cloud bills that overrun the budget are common, and the overrun usually comes from fees that weren't part of the original forecast. Line up your last three invoices. If the swings aren't tied to a deliberate scaling decision, that's not a pricing model, it's a forecasting liability finance absorbs every cycle.

9. What does contention on shared capacity actually cost you in wasted GPU time?

Model FLOPs Utilization (MFU), the share of a GPU's peak compute that a training job turns into useful work, is the metric to ask for, and on production jobs it sits well below the spec sheet. Checkpointing, initialization, queue delays and debugging add idle time on top. None of that shows up as its own line item. It shows up as jobs that take longer than they should. Measure both on your own jobs and set them against the GPU hours on the invoice.

10. What would your trailing 12-month cloud GPU spend total on a fixed-rate, dedicated contract instead?

Add your base compute, egress, storage, support, and overrun charges from the last 12 months, then compare that total with a written proposal for a fixed monthly rate over the same period. Run the comparison on sustained workloads rather than short bursts: a fixed rate is the same whether the servers are busy or idle, so it suits capacity you keep in use.

Individually, none of these questions are hard to answer. Together, they tend to point to the same conclusion: the real cost of cloud GPU infrastructure was never the number on the pricing page. It's the accumulation of egress, idle capacity, storage sprawl, and support fees that each look small in isolation and add up to a materially different number over a full contract term.

Dedicated bare metal infrastructure is built around removing that accumulation. A single server, a fixed monthly rate, no egress fees, and no line items that surface only after the fact. At 1Legion that means whole servers on 36, 48 or 60 month terms, a down payment at signing and one fixed monthly rate for the full term, as set out in how to buy. It isn't the right architecture for every workload, but for teams running sustained AI or media production at scale, the math above is worth running before the next renewal is signed.

Talk to an engineer about what a fixed-cost, dedicated GPU setup would look like for your workload.