TL;DR
- Your serving engine has no built-in way to snapshot GPU memory, so every new replica builds the same state from zero and you wait minutes for it.
- Three products besides Cedana save that state on a running inference worker and bring it back: NVIDIA Dynamo Snapshot, Modal's GPU memory snapshots, and InferX. You do not have to move to a hosting platform to get one, but you do have to read what each of them covers today.
- NVIDIA Dynamo Snapshot is experimental and single-GPU today. Modal's is alpha and "generally incompatible with multi-GPU code". InferX publishes one GPU snapshot number and no compatibility rules at all. So each product covers less than the mechanism suggests.
- A snapshot removes the work of building the state, not the time to read the weights off storage. So it helps you in proportion to how much of your wait is initialization.
- In this piece we walk through what NVIDIA Dynamo Snapshot, Modal and InferX each restore today, and the cuda-checkpoint and CRIU plumbing underneath them. Then we cover the CRIU-based products that leave GPUs out, and what Cedana restores.
All three products are recent, so each one publishes a short statement of what it covers today. The table sets those statements next to each other in the vendors' own words.
| Option | What it restores | Where it can restore to | What it covers today | Published limits | The published number |
|---|---|---|---|---|---|
| NVIDIA Dynamo Snapshot | The process tree from CRIU, plus the CUDA device state | "a different node" | "single-GPU vLLM and SGLang workloads" | Experimental; the CRIU speed-ups "are not yet shipped"; the GPU Memory Service "gated on pending CUDA driver patch"; TensorRT-LLM, multi-GPU and multi-node on the roadmap | "under 5 seconds" on the gated path |
| Modal GPU memory snapshots | CPU memory plus "Device memory contents (GPU vRAM), such as model weights" | Another host "with compatible GPU hardware" | Functions on Modal, alpha | "generally incompatible with multi-GPU code"; snapshots "do not speed up model loading from storage"; code rewrites; "2-3 snapshots per GPU type", invalidated on redeploy | vLLM on Qwen2.5-0.5B-Instruct, "45s (P0)" to "5s (P0)" |
| InferX | "CPU, CUDA, and multi-GPU communication state" | Migration is listed; no compatibility rule published | "production inference workloads on demand", multi-GPU claimed | None published | "GPU Snapshot Restore <1s", one configuration, about 70 GB in FP16 on an H100 |
| Cedana | The running process with its GPU memory, below the serving engine | The same node or another compatible node | Single GPU and multi-GPU on one node, under SGLang, vLLM or TensorRT-LLM | Same GPU model, driver, engine and model versions, or the checkpoint is invalid. Single node today | 57 to 70 seconds to restore, against 564 to 2,051 seconds to cold start, on 8 B200s |
What NVIDIA's experimental release supports today
NVIDIA Dynamo Snapshot was published on 27 May 2026, and it does checkpoint and restore for inference workers on Kubernetes. It saves the worker in two halves, the GPU side and the host side. For the GPU they "use the checkpointing capability of the CUDA driver (which is also exposed by the cuda-checkpoint command line tool) to dump the device state to CPU memory". For the host they use CRIU to "serialize the process tree's state to disk", and they can "fetch the checkpointed artifact from a different node", which is what lets the restore happen somewhere else.
NVIDIA cuts the artifact down by taking the snapshot before the replica has served a request, so "this KV cache buffer does not need to be checkpointed at all", and dropping it takes a Qwen3-0.6B artifact on a B200 "from ~190 GiB to ~6 GiB".
What ships is narrower than the mechanism, and NVIDIA states the window in one sentence: "Today, the experimental release supports single-GPU vLLM and SGLang workloads through the non-GMS checkpoint/restore path." The CRIU speed-ups behind their fastest restore times "are not yet shipped as part of Dynamo Snapshot", and the GPU Memory Service, the path that reaches "under 5 seconds", is "currently gated on pending CUDA driver patch". TensorRT-LLM, multi-GPU, and multi-node are named as roadmap rather than product, so if your model spans more than one GPU, this release does not cover you yet.
NVIDIA's documentation link was broken when we checked, so what ships now may differ from what the blog describes.
What Modal says GPU memory snapshots do not do
Modal offers the same mechanism as a flag on a serverless platform, which means you get it by running your inference on Modal. Its description separates the two halves: "CPU Memory Snapshots capture the state of CPU memory. GPU Memory Snapshots, an alpha feature, also capture the state of GPU memory."
A snapshot can come back on a different machine, because Modal handles compatibility "across different worker hosts, ensuring snapshots created on one machine can safely restore on another with compatible GPU hardware." The result Modal publishes is for one small model: "vLLM running Qwen2.5-0.5B-Instruct would previously take 45s (P0) to startup and now takes 5s (P0)."
Multi-GPU code is out, because "GPU Memory Snapshots are generally incompatible with multi-GPU code". Your own code usually has to change too, since "most Functions will need some of their code rewritten", and that is "particularly true for more complex inference engines". The snapshots also go stale, because "GPU Functions need 2-3 snapshots per GPU type" and a redeploy invalidates them.
Modal's storage limit applies to every product on this page, including Cedana: "GPU Memory Snapshots do not speed up model loading from storage." If most of your wait is weight loading, snapshots "will generally not improve your cold start times, and may even worsen them". What a snapshot removes is "library initialization (imports) and JIT compilation (Torch, DeepGEMM, Triton, etc.)", so the number worth having before you choose anything on this page is how much of your own start-up is initialization.
InferX claims multi-GPU restore and publishes no limits
InferX makes the widest claim here, and it sells a GPU runtime built on snapshot restore. "InferX virtualizes CPU, CUDA, and multi-GPU communication state to snapshot, restore, migrate, and scale production inference workloads on demand." Its architecture page lists "Multi-GPU Snapshot/Restore" and "Secure NCCL Isolation" as features, and says the runtime works "By intercepting CUDA and NCCL interactions".
The one published figure is "GPU Snapshot Restore <1s", and the benchmarks page attaches it to a single configuration, a Qwen3.6-35B-A3B model in FP16 at about 70 GB on an H100. The other half of the picture is missing, because InferX's pages publish no failure modes, no unsupported cases, and no compatibility matrix. So the claim reaches further than NVIDIA's or Modal's, and you have less to check it against.
NVIDIA and Modal build on cuda-checkpoint and CRIU
Under all of this sit cuda-checkpoint, which saves the CUDA state of a Linux process, and CRIU, short for Checkpoint/Restore In Userspace, which freezes a running process and saves its host-side state to disk. NVIDIA Dynamo Snapshot uses the two together to checkpoint and restore an inference worker.
More than one product on this page uses them, so what separates the products is the layer each one builds on top. That layer decides when a snapshot is taken, captures the state around the GPU, holds every GPU of one job at the same point, and controls where the artifact can land.
Three other products built on CRIU leave GPUs out, so none of them is the third-party tool you are looking for. Loophole Labs Architect says in its FAQ that "GPU workloads are not supported yet". Cast AI's limitations page says GPU workloads are "Not currently supported due to the complexity of checkpointing and restoring GPU memory and compute". Neither GPU nor CUDA appears in zeropod's README.
What Cedana restores today
We build that layer at Cedana, and we put it below the serving engine, at the operating-system and CUDA-driver levels. On the GPU side we interpose at the driver API, and on the host side the daemon hands the process to CRIU. What comes back is the process with its GPU memory and the state around it, captured the same way under SGLang, vLLM, and TensorRT-LLM.
The single-GPU and multi-GPU-on-a-single-node tiers ship in production today. Multi-node, where a single workload spans hundreds or thousands of GPUs across many nodes, is in design partnership with leading enterprises and neoclouds. A job on the 8 GPUs of one node can be saved there and restored on the 8 GPUs of another, because at no point was it running on two machines at once.
Your restore has to land on compatible hardware. A checkpoint records the GPU, driver, engine and model versions it was taken against. If any of them changes, the checkpoint is invalid and the workload cold-starts instead.
Our published figures come from one node of 8x NVIDIA B200 GPUs with 1.7 TB of system memory on CUDA 12.9, running four frontier models under SGLang. Native cold starts ran 564 to 2,051 seconds, and restores of the same fully initialized engines ran 57 to 70 seconds. Those checkpoints were read from tmpfs, the node's own memory, so a read from disk or across a network is not in the figures. Restore time follows the size of the checkpoint, meaning how many bytes have to move, not the parameter count.
The storage constraint Modal describes applies to Cedana as well. A checkpoint does not make the weights load from storage any faster, because what it removes is the need to build the state a second time. The saved bytes still have to move, and somebody pays one full cold start to make the checkpoint in the first place.
NVIDIA proved the mechanism works and published what its experimental release covers. We work at the same two layers, under whichever of those engines you serve with. Cedana is automated GPU checkpointing and migration infrastructure that increases the useful work your GPUs deliver. On this page that means our row of the table: single GPU and multi-GPU on one node, restoring on the same node or another compatible one. If your workload fits on one node, that is a checkpoint you can take today. If it spans nodes and that is the case you need solved, talk to Cedana about a design partnership.
Every quotation is from a source named above, read in September 2026.
Related:
- Where the minutes go when an LLM worker cold starts
- Can LLM inference scale to zero without paying for warm replicas?
- How to swap models on one GPU without a cold start
- How Cedana works: checkpoint, restore, and migration below the serving engine
- Does disaggregated serving remove the need to move GPU workers?
- Your KV cache now rivals your weights
- KV-cache offload versus a saved worker: what each survives
Common questions
My inference engine has no built-in way to snapshot GPU memory for fast cold starts. Do I need a hosting platform like Modal or InferX, or a third-party tool, to get that?
You do not have to move to a hosting platform, but you do need something outside the engine, because the snapshot is taken below it. Three products besides Cedana do this today. NVIDIA Dynamo Snapshot's experimental release supports single-GPU vLLM and SGLang workloads and can restore to a different node. Modal's GPU memory snapshots restore functions between hosts with compatible GPU hardware, in alpha, and they come with running your inference on Modal. InferX claims multi-GPU restore under one second on one published configuration, but publishes no compatibility matrix or failure modes.
My serving engine takes minutes to start. How fast do these snapshot products claim to make that?
Each product publishes its number for a different setup, so read each one next to its conditions. NVIDIA's experimental release claims under 5 seconds on a path still gated on a driver patch. Modal, in alpha, reports vLLM on Qwen2.5-0.5B-Instruct, a small model, going from 45 seconds to 5 seconds. InferX publishes a GPU snapshot restore of under one second on one configuration, a 35B model on an H100. Cedana's own measurements ran 57 to 70 seconds to restore against 564 to 2,051 seconds to cold start, on 8 B200s.
