TL;DR
- When a GPU dies under a running job, the work in flight dies with it, and you have two ways to plan for that. One is a hot standby that takes over at once. The other is checkpoint recovery, which saves the job's state while it runs and restores it on healthy hardware.
- The standby is what you buy when a few seconds of outage is unacceptable. It costs a full second machine that earns nothing until the first one dies, and it does not bring back the work the dead node was doing.
- Checkpoint recovery buys no second machine. It costs a running overhead on the node already working, plus storage for the saved state. It brings the workload back in tens of seconds instead of the minutes a cold start takes.
- In this piece we walk through what a hot standby buys, what a second 8-GPU node costs at list price, and what each answer saves when the node dies. Then we cover what GPU checkpointing costs while the workload runs, and how to decide this tier by tier.
A hot standby takes over fast enough that most users never notice
Hot standby predates GPUs, and the pattern comes from general failover work rather than from anything GPU-specific. Aerospike's explainer on failover mechanisms gives the established definition: one or more backup systems kept fully operational and synchronized in real time with the primary, so that when the primary dies the standby assumes the role immediately, often with no downtime a user would notice. For a GPU serving tier, the standby priced here is a second node with the model loaded and ready to take traffic, and in that configuration the second node does not mirror the primary worker's in-flight sessions.
This is what you buy for a service where a few seconds of outage is unacceptable, and it works. No restore of a frontier-scale worker in our published set is that fast, because the standby is already running and has nothing left to load.
A warm standby is the cheaper arrangement in between, kept partly running and updated from the primary, so it fails over in minutes rather than at once. The comparison below covers hot standby and checkpoint recovery.
The standby costs a second machine that earns nothing until the first one dies
What you pay for that takeover is a full duplicate of the system you are protecting, running in parallel at all times, which doubles the infrastructure for the sake of redundancy. This comparison prices both answers on one 8-GPU node, and a public price list puts a number on the standby side of it.
At Lambda's list price dated 1 September 2026, the 8-GPU B200 on-demand tier was $6.69 per GPU per hour, excluding tax. Eight GPUs at that price is $53.52 an hour, so a second node of the same shape is about $53 an hour, and over a 730-hour month the arithmetic gives roughly $39,000. That is an on-demand list price rather than what you would pay under a reservation, and pricing pages move, so check the row on the day you build the plan.
The second node earns nothing while the first one is healthy, so you are holding a machine against an event that may not come this month. None of that makes a standby the wrong buy, because limiting customer-facing downtime is what the spend is for.
What each answer saves when the node dies
A serving worker holds state that lives only in GPU memory, and for every request in flight that state is the work the machine has already computed. If nothing has saved that state, the only way to get it back is to compute it again.
The MLSys 2026 paper GhostServe describes the consequence for fault-tolerant serving: "In the event of a failure, this volatile state is lost, forcing the system to restart the inference job from the very beginning."
A standby does not save that state. Traffic moves to a machine that is already running, but a machine being ready to serve does not mean it holds the failed worker's session state, so the standby replaces the hardware and the work the hardware was doing is gone with it. Checkpoint recovery is different in one respect. It restores the workload state saved at the last checkpoint, so what the worker held then comes back, and the work since that checkpoint is repeated.
Checkpoint recovery spends on the node already working
Checkpoint recovery buys no second machine. It adds overhead to the node already doing the work, because saving state while a job runs takes time and bandwidth away from the job. Microsoft Research, in its work on just-in-time checkpointing, reports that periodic checkpointing carries significant steady-state overhead on large models.
How large that overhead is depends on your workload, how often you save, and where the bytes go, so measure it on your own cluster rather than reading it off a vendor page. Ask any vendor selling this, including us at Cedana, for one figure with its conditions: what the checkpointing overhead is on a single GPU and on multiple GPUs, on which workloads, and how it was measured. A number without those conditions cannot go in a plan.
Storage is the other half of that cost, because the saved state has to go somewhere. In our published 8x B200 set the checkpoints run 244 to 873 GiB, so this side of the comparison buys disk and bandwidth where the standby buys a second GPU node.
What the overhead and the storage buy you is a restore in seconds instead of a rebuild in minutes. Our published benchmark restored frontier models, meaning the latest open-weight models trailing the frontier labs, in 57 to 70 seconds on a single node with 8x NVIDIA B200 GPUs. The checkpoints were held in tmpfs, the node's own memory, so a fetch from disk or across a network is not in those seconds. Native cold starts of the same models on the same node ran 9.4 to 34.2 minutes.
Both measurements end at the moment the service can answer a request. The cold start clock starts at engine launch and includes weight loading and full engine initialization, and the restore clock starts when the restore of the same fully-initialized engine begins.
Restore time follows the size of the checkpoint, meaning how many bytes have to move, not the parameter count. So the number you plan with tracks the bytes in your checkpoint, and a bigger model next year does not by itself raise it.
Checkpoint recovery does not need a machine reserved in advance
The two answers cost different amounts because they reserve different things. A standby reserves a machine whether or not you need it, while checkpoint recovery reserves only the state and then uses whatever healthy machine of the same shape is free when a failure occurs. A GPU failure triggers a restart on a healthy node.
Decide this tier by tier rather than once for the whole fleet
Buy the standby for services whose users notice even a few seconds of downtime. Use checkpoint recovery wherever the work can wait for a restore of about a minute, plus the time to notice the failure and get a free machine. Some work needs neither, because if a job restarts in seconds the retry costs less than anything you would buy to avoid it.
| Criterion | Hot standby | Checkpoint recovery |
|---|---|---|
| What you buy | A second machine with the model already loaded, running in parallel, ready to take traffic at once | The workload's state, saved while it runs and restored on healthy hardware |
| What it costs while nothing is failing | About $53 an hour for a second 8-GPU B200 node at the 1 September 2026 list price, excluding tax | A running overhead on the node already working, plus storage for the saved state |
| How long the takeover or recovery takes | Immediate, and users often notice nothing | The restore itself ran 57 to 70 seconds for frontier models in our published 8x B200 set. The outage also includes the time to detect the failure and find a free machine |
| What happens to the work in progress | Lost with the node; the standby holds none of it | Restored to the state saved at the last checkpoint; work since then is repeated |
| What has to be true about the target hardware | A machine of the same shape is reserved and running before the failure | A healthy machine of the same shape is free at the moment of the failure |
| What else the same spend covers | Nothing else. The spare exists for the failure | The same saved state also serves a planned move or a maintenance drain |
- If you run an inference platform or a neocloud, this decision sets the spare capacity line in your budget, because a restore that finishes in tens of seconds turns the spare from something you hold across the fleet into a choice you make per tier.
- If you run your own coding models, ask what a minute costs the developers using them: a minute that passes unnoticed makes checkpoint recovery the cheaper answer, and if every engineer feels it, the node is worth its price.
- If you run a research cluster, a spare is not usually in the budget, so the standby is rarely something you can buy, and your question is instead what saving state costs on your own workloads, which is the number to make a vendor produce.
Saving state does not make failures rarer. It shortens the outage that follows one, so a tier where that wait shows up on a customer's screen still needs a standby of its own. Everywhere else, what you need is something that saves the state of a running workload and brings it back on healthy hardware. Cedana is automated GPU checkpointing and migration infrastructure that increases the useful work your GPUs deliver. In this comparison, that is the recovery column: the state is saved while the job runs, and the restore lands on whatever healthy node of the same shape is free. You get the shorter outage without paying for a machine that waits.
Related:
