TL;DR
- You are holding a quote for GPU checkpointing, and finance wants to know what has to be true before the fee pays for itself.
- Vendors price the waste rather than the fix, so their case does not answer a team weighing the engineering time against work on the product.
- The break-even is the annual fee per GPU divided by what one GPU costs you to run for a year. The number of GPUs cancels out, so a 50-GPU fleet and a 500-GPU fleet need the same percentage-point gain.
- Two things move that number: the price per GPU on your quote and your own all-in cost per GPU-hour. At a $2,000 annual fee and $4.00 per GPU-hour the gain needed is 5.7 percentage points.
- In this piece we walk through where that division comes from, how to find the cost of an hour of your own GPU time, and why fleet size does not change the answer. We also cover what does change it, and which hours the gain would have to come from.
Teams already weigh this against product work
The objection usually arrives before the price does, and it is about engineering time rather than money. A platform team running about 128 GPUs wrote in an r/MLOps thread about idle GPUs that they care less about effective utilization than about having cost-efficient capacity when they need it, because "the engineering time to optimize utilization is often a wash between optimization (less expenses) and improving our product (more revenue)".
Civo, writing in June 2026, puts the vendor case in money: a GPU running at 30% utilization costs the same as one running at 90% while doing a third of the useful work. That is true, but it settles nothing for the team above, because it prices the waste without pricing the fix.
The break-even is the annual fee per GPU divided by what one GPU costs you to run
The fee is on the quote in front of you, and the cost of running one GPU for a year belongs to whoever owns the cluster's operating budget. The fee on the quote is the license, so if the engineering time to run it is material to you, add that time to the fee before you divide.
Break-even in percentage points = 100 x annual fee per GPU / (your all-in cost per GPU-hour x 8,760)
Paying for a GPU all year means paying for 8,760 hours, so the denominator is your hourly cost multiplied by 8,760. What comes out is the additional share of those hours, in percentage points, that must produce completed work for the fee to be covered. If your quote is priced against the memory under management rather than per card, convert it to an annual fee per GPU first.
Find the cost of an hour of your own GPU time
The quote gives you the fee, so the figure you still have to find is what an hour of your own GPU time costs. Renting hands it to you, because a rate card states it. GMI Cloud's pricing page, as of September 2026, lists the NVIDIA H100 from $2.00 per GPU-hour and the B200 from $4.00, and Lambda's on-demand page prices the H100 at $3.99. Those rates carry the provider's margin, so if you rent, the applicable rate is your cost.
A rate can also be a promotion with an end date on it. Together AI's pricing page, in the same month, shows the H100 at a promotional $3.99 against a list price of $5.49, and dates the promotion to expire on 30 September 2026, so read the card on the day you do the arithmetic.
If you own the hardware, you have to build the figure yourself. It is the all-in operating cost, meaning electricity, cooling, installation, depreciation, and the staff who keep the cluster running. We put that at about $1.40 to $1.50 per GPU-hour for a hyperscaler and $3 to $5 for an enterprise, depending on how efficient the operation is. That range is Cedana's estimate rather than a published figure, and the spread is wide, so use your own finance team's number if you have one.
Whichever figure applies, multiply it by 8,760 to get what one GPU costs you for a year. That is $17,520 at $2.00 an hour, $35,040 at $4.00, and $43,800 at $5.00. A fleet that is powered down for part of the year has a smaller denominator and a higher break-even, so count the hours you pay for.
A 50-GPU fleet and a 500-GPU fleet need the same gain
Take an operating cost of $4.00 per GPU-hour, in the middle of the enterprise range, and a fee of $2,000 per GPU per year. We publish no price, so the fee in this table and in the next one is a placeholder you replace with the figure on your own quote. For each fleet, multiply both the annual operating cost per GPU and the annual fee per GPU by the GPU count.
| Fleet | GPUs | Annual operating cost of the fleet | Annual fee, placeholder | Break-even in percentage points |
|---|---|---|---|---|
| Smaller | 50 | $1,752,000 | $100,000 | 5.7 |
| Larger | 500 | $17,520,000 | $1,000,000 | 5.7 |
The bigger fleet's two dollar amounts are 10 times larger, and its break-even is the same 5.7 points, because both sides of the division scale with the number of GPUs and the count cancels. That holds wherever the price per GPU is the same at both sizes.
Two things move the break-even, and the count is not one of them
The price per GPU is the first. A quote covering more GPUs usually carries a lower price per GPU, so a larger fleet can land on a lower break-even. The lower break-even comes from the price, and the count still divides out.
Your operating cost per GPU-hour is the second. The higher it is, the more each recovered hour is worth to you, so the lower the break-even falls. A cluster costing $5.00 an hour for each GPU breaks even on fewer percentage points than the same cluster at $2.00 for the same fee, which is why an enterprise and a hyperscaler reading one quote get different answers.
| Annual fee per GPU, placeholder | At $2.00 per GPU-hour | At $4.00 per GPU-hour | At $5.00 per GPU-hour |
|---|---|---|---|
| $1,000 | 5.7 | 2.9 | 2.3 |
| $2,000 | 11.4 | 5.7 | 4.6 |
| $4,000 | 22.8 | 11.4 | 9.1 |
Every cell shows percentage points of the fleet's annual hours, calculated from the placeholder fee in its row and the hourly cost in its column. A gain of ten percentage points breaks even only in the cells whose required gain is at or below ten points, so it does not reach the 11.4- or 22.8-point thresholds. These fees illustrate the calculation, so use your own quote and your own operating cost rather than reading them as Cedana's price list.
A recovered hour is worth what an hour already costs you
What the cluster is for decides whether the gain reaches you as money coming in or as money not going out. An operator who sells GPU-hours or tokens can bill a recovered hour, so the gain arrives as revenue on hardware already bought. On a cluster run for your own researchers or your own product, nobody pays you for the extra hour, so the gain there is a cost avoided: the work fits inside the hours you already pay for instead of requiring hours you have not bought.
Either way, the division is a payback test rather than a revenue forecast. It values the hours you get back at what an hour already costs you, and it says nothing about what those hours will earn.
The points come from hours you already pay for that finish no work
Effective utilization is the share of your GPU-hours that produced work you kept, and it is the number the break-even is measured against. A dashboard reports something else, because it counts a GPU as busy while a model loads and while a failed job recomputes what it had already done. A held allocation reads as busy in the same way, whether or not anything is running inside it.
We see allocated GPUs often running below 40% utilization, and some near zero, though those are raw utilization figures rather than the effective share. Failure recovery is the single largest category of the hours that finish no work, larger than over-allocation, maintenance windows, and cold starts combined. That ranking is our own observation rather than a published measurement, so use your job accounting to measure how many hours each category consumes on your cluster.
Total your GPU-hours by the state each job ended in. The share spent on jobs that never completed is the part failure recovery would address, and if that share is already larger than your break-even, the question left is how much of it would be recoverable on your own workloads.
Not all of those hours come back. A restore costs the restore time plus the work done since the last checkpoint, so subtract both from the hours you expect to recover. Checkpointing on an interval while the job runs, which we call heartbeat checkpointing, is live at Cedana, and after a failure a workload resumes automatically on a new node. A move for any other reason, such as freeing a node, works differently, because we do not decide where a job goes: an operator or an agent starts that move under a policy you write.
Whether those hours can be recovered at all depends on the hardware the job lands on and on how many nodes the workload spans. A checkpoint records the GPU, driver, engine and model versions it was taken against. If any of them changes, the checkpoint is invalid and the workload cold-starts instead.
The node count is the second condition. The single-GPU and multi-GPU-on-a-single-node tiers ship in production today. Multi-node, where a single workload spans hundreds or thousands of GPUs across many nodes, is in design partnership with leading enterprises and neoclouds.
Every hour you do recover comes back the same way: something saves the state of the running job and brings it back on other hardware, so the hour goes into finishing work instead of repeating it. Cedana is automated GPU checkpointing and migration infrastructure that increases the useful work your GPUs deliver. The hours it hands back are the ones that would otherwise have gone into loading a model again or recomputing a lost run, so what the fee is worth to you is that share of your hours priced at your own rate.
Related:
- Which utilization number goes into your own-versus-rent calculation?
- How to put a dollar figure on the GPU-hours that produced nothing
- Tasks per dollar: the economics of running your own coding models
- Cost per token and revenue per megawatt are the same number
- The fleet's real yield metric is tokens per gigabyte of VRAM
- Sovereignty has a bill: the transferred duties and the availability math
Common questions
Is it even worth spending engineering time to optimize our GPU utilization, or is that time better spent on the product?
You can settle it with one division. Divide the annual fee per GPU by your annual operating cost per GPU, which is your all-in cost per GPU-hour times 8,760, and add the engineering time to the fee first if that time is material to you. What comes out is the gain in effective utilization, in percentage points, that pays for the fee, and the number of GPUs cancels out, so a 50-GPU fleet and a 500-GPU fleet need the same gain. At a $2,000 annual fee and $4.00 an hour that gain is 5.7 percentage points, which a gain of ten percentage points clears.
How much is idle, queued, or unused GPU capacity costing us?
We see allocated GPUs often running below 40% utilization, and some near zero. Those are raw utilization figures rather than the effective share that produced work you kept, and the effective share is what the break-even is measured against. Failure recovery is the single largest category of the hours that finish no work, larger than over-allocation, maintenance windows, and cold starts combined, though that ranking is our own observation rather than a published measurement. To put a dollar figure on those hours at your own rate, use How to put a dollar figure on the GPU-hours that produced nothing.
