FLUX diffusion cold start: 50 minutes to 73 seconds
Diffusion pipelines pay a cold-start tax LLMs never see: compilation. With per-resolution torch.compile across 11 fixed resolutions, FLUX takes about 50 minutes to reach ready-to-serve natively on 8× NVIDIA B200. Cedana restores the same fully compiled, fully initialized pipeline in 73 seconds.
Compilation is the new cold start. Restore skips it entirely.
Native cold start scales with compilation work. Cedana restore doesn't.
8× NVIDIA B200 · FLUX with FP8 and torch.compile max-autotune · 11 fixed resolutions, per-resolution compilation · speedup computed from ~50 min vs 73s
Why diffusion model cold starts take so long: every resolution is its own torch.compile.
Production diffusion serving pins fixed resolutions for compiled performance, and each resolution triggers its own torch.compile pass. torch.compile with max-autotune can take minutes per resolution on a diffusion transformer (DiT) like FLUX. With max-autotune and an exhaustive GEMM autotune search, that work dominates startup: weights load in seconds, but the pipeline isn't ready to serve until every resolution's kernels are compiled and tuned. Eleven resolutions pushed native cold start to roughly 50 minutes in this configuration.
This is exactly the state worth checkpointing. Compiled artifacts, autotune results, and the fully initialized pipeline are all part of the workload's execution state, so a Cedana restore brings back the ready-to-serve pipeline without redoing any compilation. Restore time is independent of how much compilation work went into the checkpoint: more resolutions, more aggressive autotune, or heavier compile modes make the native cold start longer and the speedup larger, not the restore slower.
The same tax applies across diffusion and video models.
The mechanism is not FLUX-specific. Any pipeline that compiles per resolution or per shape pays the same startup tax: SDXL, Stable Diffusion 3.5, and video generation models such as Wan and HunyuanVideo all combine large weights with shape-specialized compilation, and video models add temporal dimensions that multiply the compile surface further. We have measured FLUX; the compilation mechanics transfer, and the pattern holds: the more compilation work a pipeline needs, the longer its native cold start, and the larger the restore advantage.
The exact args, reproducible.
All runs on a single node with 8× NVIDIA B200 (Blackwell). The configuration below replicates a production-style deployment with fixed-resolution compiled serving:
- FLUX_STEPS
- 4
- FLUX_DEVICE_MAP
- balanced
- FLUX_COMPILE
- 1
- FLUX_COMPILE_MODE
- max-autotune
- FLUX_FP8
- 1
- FLUX_PER_RES_COMPILE
- 1
- TORCHINDUCTOR_MAX_AUTOTUNE_GEMM_SEARCH_SPACE
- EXHAUSTIVE
We see similar results across other argument configurations: the compile settings change how long the native cold start takes, while Cedana restore stays roughly constant.
Native cold start is the time from pipeline launch to ready-to-serve at all 11 resolutions, including weight loading, initialization, and all per-resolution compilation. Cedana restore is the time to restore the same fully compiled, fully initialized pipeline from a Cedana checkpoint to ready-to-serve.