Every era of computing needed a migration primitive. GPUs are next

GPU work is stateful in a way CPU services never had to be

Somewhere in your fleet, a GPU is about to fail. When it does, the job on it will vanish rather than slow down, degrade, or fail over to a neighbor the way a web server would. The weights that took tens of minutes to load, the cache holding every conversation in flight, and the days of training progress held only in memory disappear in the time it takes a card to drop off the bus, and you start the work over from the beginning.

A CPU service survives the loss of its own machine because the work it is doing is written down somewhere else. The database holds the order, the queue holds the job, and the load balancer sends the next request to a server that is still up, so a restart reconnects to that state and carries on. That is why a small on-call rotation can run thousands of CPU servers and treat a dead one as routine.

GPU work has nowhere else to put its live state. The loaded model weights sit in GPU memory, and so do the sessions in flight and the key-value (KV) cache that holds the working state of each one. A failed card destroys all of that live state, and taking the card down for maintenance or preempting its job has the same operational effect.

The amount of state involved is what makes this a new problem rather than an old one with bigger numbers, because some current open-weight models need most or all of an 8-GPU node just to hold their weights. Kimi K2.6 is a trillion-parameter mixture-of-experts model, and its published model card and deployment guide list a serving configuration of a single H200 node with 8 GPUs. By weight-file arithmetic, its weights come to roughly 600 GB even in native INT4, so a worker holding that model's loaded weights, plus the sessions it is serving, carries tens to hundreds of gigabytes of live state with no complete copy anywhere else.

The model files persist, but no database holds a durable copy of the complete running worker. Nothing outside the GPU holds the work in progress as it stood at failure, so a retry runs that work again from the beginning instead of picking it up where it stopped.

Nobody has built a general way to pick that work up and set it down elsewhere, which is what people notice the first time they see it done. At the end of one demonstration of a live GPU workload migrating, a veteran of high-performance computing told us they had been looking for something like this for 20 years, and we have heard the same sentence, almost word for word, more than once since.

Every production layer before this one built a migration primitive

Every layer before this one ran into the same problem, state stuck to the machine that held it, and got past it by saving the live state of running work and bringing it back on other hardware. Four layers built such a thing, each in its own way, and every one of them bought the same two things, reliability and utilization.

Operating systems tried process migration first

Research operating systems went first, in the 1980s and 1990s, on clusters of workstations. Sprite, built at Berkeley between 1984 and 1992, made transparent process migration one of its headline contributions, and MOSIX, developed at the Hebrew University from 1981, moved running processes between machines on its own initiative to balance load across the cluster.

Neither reached the mainstream, and the field said so about itself: a 2000 survey in ACM Computing Surveys by Milojicic, Douglis, Paindaveine, Wheeler, and Zhou found that process migration "has not achieved widespread use."

The reason was structural: a process is entangled with its kernel through open files, sockets, shared memory, and device state, and none of that lifts cleanly out of one machine and into another. The goal was right, but the boundary was drawn in the wrong place.

Virtual machines made it work in 2003

VMware shipped vMotion with VirtualCenter 1.0 in 2003, and Mike Nelson, the engineer who built the first prototype, later described to the Yellow Bricks blog how small a step it seemed from what the product could already do.

"It was an obvious next step that if we could checkpoint to disk and resume on another machine that we should be able to checkpoint over the network to another machine and resume."

The academic version of the same idea, iterative pre-copy of memory pages, arrived two years later in Clark and colleagues' live migration paper at NSDI 2005, so the shipping product came before the published algorithm.

Before vMotion, a workload landed on a physical host when it was provisioned and stayed there until the host was decommissioned. Hardware maintenance meant scheduling downtime for the application, and moving it to a less loaded host meant an outage.

The launch demo answered all of that with a game of Windows pinball. Nelson set two machines side by side, started the game on one, migrated the running virtual machine to the other, and kept playing. The entanglement that had defeated process migration was tractable here because of where the boundary sat: everything a process depends on lives inside the machine image, so the whole image moves at once.

Keith Adams, an early VMware engineer who backs Cedana, has written publicly that VMware's suspend, migrate, and resume capability turned out to be one of the more flexible and powerful primitives the company shipped: teams could upgrade hardware or system software out from under a mission-critical service, drive higher utilization from a given pile of physical hardware, and set up automated high availability.

Containers took it into userspace

Putting checkpoint and restore inside the Linux kernel had been tried and had stalled, so the work moved outside it. Pavel Emelyanov of the OpenVZ team first presented CRIU, checkpoint and restore in userspace, to the kernel community in July 2011, and it became the mechanism for checkpointing and moving containers.

The Kubernetes project shipped forensic container checkpointing as an alpha feature in version 1.25, in December 2022, and it has since reached beta and is enabled by default.

On 21 January 2026, the project formed a Checkpoint/Restore Working Group, and three of the six use cases the group named are AI workloads:

• faster startup for applications with long initialization times, including Java applications and LLM inference services
• periodic checkpointing for fault tolerance in long-running work such as distributed model training
• better utilization for interactive workloads such as Jupyter notebooks and AI chatbots

Databases separated the state from the compute

Databases came at the problem from the other direction: instead of moving a running process, they separated the state from the machine and replicated the state.

PostgreSQL's pg_basebackup takes a base backup of a running cluster with no downtime, which is how replicas get built, and Percona released XtraBackup 1.0 in December 2009 to do the same for MySQL. The warehouses split the two apart in the architecture itself, so BigQuery launched in 2012 with compute and storage on separate clusters, and Snowflake's engine, described at NSDI 2020, runs stateless compute against storage that lives elsewhere.

So for a database, migrating a query means starting new compute against the same durable storage, and the architecture itself provides the migration primitive. A database has the one thing a GPU workload does not have: a single durable copy of the truth that the compute can be rebuilt from.

The four layers differ in everything except the shape of the fix.

The primitives that stuck all shared four properties

A GPU migration primitive needs the same four properties, and all four describe work that the layer takes off the teams above it.

1. It is transparent to the workload above it. The application does not know it moved.
2. It is built at the system layer, not inside the application. Application-level schemes exist in every era and stay niche because each application has to implement and maintain its own.
3. It works broadly across the layer rather than supporting only one framework or workload type.
4. It has an owner with the incentive to maintain it across generations of hardware. Sprite and MOSIX were research projects. vMotion outlived several generations of servers because a company was paid to keep it working.

The database answer fails for GPUs, so GPUs need vMotion's answer

The database answer fails for a GPU worker because there is no durable copy of the complete running worker to rebuild from. The state of that worker includes the model weights loaded into memory, the KV cache, the in-flight sessions, the CUDA context, and the surrounding process, file, and network state, and hundreds of gigabytes of it can sit in VRAM, the GPU's own memory, with no cheap store behind it.

Offloading the cache to storage covers less of that than the name suggests. A KV offload tier can reload only the prefixes it has already written to storage, and it cannot reload the computation that was in flight when the GPU died, because that computation had no copy anywhere, so the work runs again from the start.

So GPUs need the answer virtual machines found rather than the one databases found, which is to save the live state and move it to healthy hardware. vMotion did that for a machine image in 2003, and a GPU workload needs the same thing done for its weights, its cache, its sessions, and the process around them, all at the same instant.

Cold starts, failures, and stranded capacity are one problem

Cold starts happen because the required state cannot be pre-built. Bringing up a frontier worker means loading hundreds of gigabytes of weights, starting the runtime and the distributed communication, profiling the cache, and capturing the compute graphs, and only then does it answer the first request. None of that can be built somewhere else and carried in, so you keep warm replicas idle to hide the wait, and you pay for the idle time.

Failures are costly because live state cannot be saved. When a GPU dies, the loaded weights, the cache, and the sessions die with it, so recovery is the full start again plus the recomputation of every session that was in flight, and what you are paying for is rebuilding state that existed a second earlier.

Capacity becomes stranded when workload state cannot follow the hardware it needs. The capacity is up and powered somewhere in the fleet, but the workload that could use it is stuck where you first placed it, because moving it would destroy it. Fragmentation, load imbalance, and hardware mismatch all follow from the same fact.

Once state can move, these three problems become ordinary operations. A cold start becomes a restore, a failure becomes a resume, and a stranded placement becomes a scheduling decision instead of a sunk cost.

Cedana's published benchmark on 8x B200 measures the difference on a cold start: Kimi K2.6, a trillion-parameter model with a 670 GiB checkpoint, resumed in 63 seconds, compared with a 20.3-minute native start on the same hardware.

The GPU primitive is late for a structural reason, not an oversight

Three approaches to saving GPU work exist today, and each stops short of those four properties in its own way.

The first is application-level checkpointing, which has been with us in every era and still is. A training framework writes its own checkpoints, and a scientific program like GROMACS has done so for decades, but each application implements and maintains its own scheme. Coverage is much better for training than for serving, and someone still has to act when a job fails, so the result repeats the operating-system era's outcome: application-specific schemes rather than a general migration layer.

The second is a kernel-level primitive, which exists for single GPUs. One of them can pause a process on a GPU, copy its device memory to the host, and hand it to a CPU checkpointer, which is a useful building block. What it leaves to you is everything around the process: the file system, the network state, the scheduler's view of the job, and coordination across the 4, 8, or 16 GPUs of a modern worker. Deciding when to act is yours as well.

The third is what the frontier labs did, which was to build application-level systems for their own infrastructure at a cost nobody below them can justify. Meta's MAST and Google's Orbax are the public examples, and they are carve-outs rather than products the rest of the market can buy.

The primitive that fits the pattern is system-level, transparent to the workload, coordinated across the GPUs and nodes of one job, and aware of the scheduler above it. It has been missing for the same reason process migration stayed niche in the 1980s: the boundary is hard.

A modern inference worker spans anywhere from 4 to 16 GPUs on one node, and those GPUs coordinate through NVLink and through frameworks like NCCL (the NVIDIA Collective Communications Library) or MPI (the Message Passing Interface). Capturing that state cleanly means synchronizing every GPU at the same point in the distributed computation, so the state saved on GPU 3 agrees with the state saved on GPU 6.

Saving the state of one GPU is a hard problem, and it has been solved. Coordinated capture across a whole worker is materially harder, and that coordination decides whether a checkpoint can be restored at all. The hard part is coordination, not capture.

Migration is what gives a workload portable state. A worker with portable state becomes something a scheduler can place, move, pause, and resume like any other object it manages, instead of a thing pinned to the silicon it started on.

The value lives above the primitive, in the control plane

The primitive sits at the bottom of a stack of three layers, and the value arrives at the top.

The bottom layer is the primitive itself, which is checkpoint and restore of the GPU state plus everything needed for a real restore: the local file system, the scheduler's view of the job, and the network connections. A saved GPU context on its own does not bring a workload back cleanly.

The middle layer is the control plane, which decides when to use the primitive. It resumes a worker automatically when one fails, enforces a service level by policy, and scales a service up and down by checkpointing and restoring workers. It also pauses and resumes jobs by priority, so one fleet can run a queue of work at different urgencies on the same hardware. This is the layer where you write a rule once and have it enforced across the fleet without an engineer in the loop.

The top layer is the outcomes, measured in whatever units your business runs on. A neocloud manages against revenue per megawatt, a team serving models against tokens per dollar and per watt, and a team running its own coding models against completed coding tasks per watt. The primitive does not improve any of those metrics on its own. The control plane built on top of it does.

VMware's history is the build order

vMotion in 2003 was not the product that made VMware. What made VMware was the decade of operator-facing automation built on top of it: distributed resource scheduling, automated high availability, and maintenance mode. The primitive came first, the management plane that decided when a machine should move came second, and the automation operators paid for came third.

Keith Adams, who watched that arc from inside VMware, has written publicly that the same arc is starting for GPUs, and that the real complexity sits in the GPU support. A checkpoint primitive on its own does not decide when to act.

So even with a perfect checkpoint primitive, a fleet still needs the layer above it, the one that senses a failure, applies the policy, and drains a node before maintenance. The same layer fills an idle window with work that can be evicted without loss, and it resumes that work somewhere else.

That control-plane layer is what we have been building at Cedana. Cedana is automated GPU checkpointing and migration infrastructure that increases the useful work your GPUs deliver. When a GPU fails or a node enters maintenance, the system moves the work to healthy hardware instead of destroying it. The GPU that is about to fail somewhere in your fleet becomes a job that moves rather than a job that vanishes.

GPU operations feel hard because the era has not produced its primitive yet

When you ask why GPU operations are so much harder than running the CPU fleet ever was, the answer is not that you missed a tool you should have found. The era has not yet produced the primitive that every earlier layer needed before its operations settled down.

Operating systems, virtual machines, containers, and databases each got theirs once the layer below them carried enough production work that restarting from scratch stopped being acceptable. GPUs crossed that line when inference became stateful and a single worker became half a terabyte of state, and the primitive is arriving now, in the same order it arrived last time.

Today your fleet can start workers and it can kill them, but it cannot move them.