For digital-native & consumer tech

Cold starts in seconds,
not minutes.

Q: We've built our own warm-pool optimization. Why Cedana?

Most platforms have. None scales across the model catalog as model sizes grow. Cedana captures at the OS level, not the application or model layer, so the whole catalog is covered and your team stops maintaining caching code.

Q: What's the steady-state overhead during inference?

Low single-digit percent on representative workloads. We show live overhead on the call.

Q: Will it work with our autoscaler and orchestrator?

Yes. Cedana works with Slurm, Kubernetes, Kueue, Ray, Armada, and Nvidia Dynamo. Cedana plus Dynamo is the marketed product form factor for inference at scale. See the docs.

Q: How does it handle our serving framework (vLLM, SGLang, Triton)?

Cedana captures at the OS level, so the serving framework is irrelevant to the checkpoint layer. We show resume across vLLM and SGLang on the call.

Q: What about compliance and data handling at our scale?

Cedana operates on GPU memory state inside your existing security boundary. It is SOC 2 compliant, ships on-premise, and supports fully air-gapped deployments.

Q: Does it handle distributed multi-GPU inference?

Yes. Distributed multi-GPU coordinated checkpoint is the differentiating capability. We show an 8-GPU model-parallel job restored as one unit on the call.

Inference is your fastest-growing line item, and warm replicas are the tax. Cedana makes cold start fast and predictable, bounded by VRAM bandwidth rather than model size, so warm pools shrink and autoscaling finally keeps up.

Schedule a demo

cold-start floor

<30s

single-GPU serving

8x GPU restore

<80s

distributed, model-parallel serving

faster cold start

up to 9.6×

Nous-Hermes 34B, leading inference platform

code changes

transparent under vLLM, SGLang, Triton

The workloads · served from a warm pool

The GPU runs behind your product, ranked by how many of you run them.

Each one keeps weights and a KV cache in GPU memory and is served from a warm pool today. Cold start is what forces that pool to sit idle.

DLRM · two-tower retrieval · deep ranking

Recommendations & feed ranking

The largest GPU inference workload at consumer scale: every feed, marketplace, and stream ranks candidates in real time, with hundreds of model variants in production.

the taxServed 24/7 under a hard p99 latency SLA. Warm pools sized for peak-of-peak sit idle off-peak, and you pay for them around the clock.

with cedana

Restore a warmed ranker in seconds, not minutes
Scale replicas to traffic instead of peak-of-peak
A failed replica restores without dropping the SLA

A/B & bandit arms · shadow deploys · per-cohort models

Experimentation & model-variant serving

Every ranking and rec surface runs many model variants at once — A/B arms, shadow models, and per-cohort versions — and each one needs a live replica to take its slice of traffic.

the taxDozens of concurrent variants, traffic split live. A warm replica per arm and cohort multiplies idle GPUs across every experiment.

with cedana

Restore any variant on demand instead of warm-pooling them all
Run shadow and holdback models without standing GPUs
Promote a winning variant with no cold-start gap

CTR / CVR prediction · real-time bidding

Ad ranking & real-time bidding

Bid and rank within a millisecond budget; a cold replica during a spike is lost auctions and lost revenue.

the taxMillisecond budget, revenue-critical. Cold start during a traffic spike means dropped bids and lost revenue.

with cedana

Predictable restore keeps the auction inside budget
Add capacity for traffic spikes in seconds
Recover a failed bidder before the deadline

Llama 405B · DeepSeek V2 · Qwen · vLLM / SGLang

Generative AI product features

Self-hosted LLMs for assistants, summarization, writing help, and in-product agents. Big weights and a large, stateful KV cache live on the GPU.

the taxMinutes-long native cold start; the KV cache is huge. Every model-size upgrade compounds the warm-pool tax at the same query volume.

with cedana

A predictable cold-start floor, 30s single-GPU / 80s 8-GPU
Snapshot a warmed model; restore on demand
The next model upgrade stops inflating the warm pool

multimodal text / image / video / audio classifiers

Trust & safety / moderation

Classify uploads at scale across modalities, with bursty, event-driven volume that is hard to predict.

the taxRuns at upload scale, bursty and event-driven. You hold warm capacity for spikes you cannot forecast.

with cedana

Burst capacity on demand instead of standing idle
Restore a classifier in seconds during a spike
Same snapshot runs on cheaper burstable capacity

visual search · image / video generation · AR

Visual & multimodal understanding

Large vision and diffusion models power visual search, tagging, generation, and camera features, all GPU-bound and latency-visible.

the taxGeneration is GPU-bound and user-visible. A warm pool per feature multiplies idle GPUs across the catalog.

with cedana

Restore large vision models fast on demand
Scale generation capacity to real usage
No idle warm pool per visual feature

real-time scoring · graph & sequence models

Fraud, risk & abuse detection

Score transactions, signups, and sessions in real time, with demand spiking exactly when fraud does.

the taxLatency-critical and bursty (checkout, signups). A cold replica during a spike is missed fraud or blocked good users.

with cedana

Add scoring capacity in seconds during a spike
A failed scorer restores before requests time out
Run safely on preemptible, cheaper capacity

candidate ranking · send-time optimization · re-engagement

Notifications & push ranking

Deciding what to send to whom and when — ranking notification candidates per user and predicting the best send window — in huge bursts that must clear before each delivery wave.

the taxMassive bursts tied to scheduled send windows. Standing capacity sized for peak send windows sits idle between them.

with cedana

Burst scoring capacity for each send window, then release it
Restore a scorer mid-batch instead of restarting the wave
Run the batch safely on cheaper preemptible capacity

recsys retraining · LLM fine-tuning · NCCL / RDMA

Training & fine-tuning on first-party data

The non-serving workload sharing the same fleet: long, distributed retraining and fine-tuning on fresh interaction data.

the taxLong distributed runs across many GPUs. A failure near the end loses hours of fresh-data training.

with cedana

Coordinated multi-GPU checkpoints (NCCL/RDMA)
Resume near-final runs after a failure
Use spot safely for long fine-tuning jobs

two-tower · retrievalscoring…

dot-product similarityANN over millions of items

gpu · 4× H100SXM5 · NVLink

util█████████████░░░82%

vram█████████████░░░66/80GB

The problem · why inference overpays

Six ways cold start inflates the bill.

WARM.TAX

The warm-replica tax

Pools sized for peak-of-peak sit idle most of the day. Industry-wide, idle warm replicas can be 20-80% of inference spend.

SIZE.SCALE

Cold start scales with model size

A 70B rollout costs more warm pool than a 13B at the same query volume. Every model upgrade compounds the tax.

AUTOSCALE.LAG

Autoscaling can't keep up

Minutes-long cold start means you cannot scale into a spike in time, so you overprovision instead of scaling.

DIURNAL.IDLE

Diurnal & event traffic idle

Traffic swings several-fold across the day and around events. A fixed warm pool pays for every trough.

BLAST.RADIUS

Failure blast radius

A failed replica drops in-flight requests and its KV cache. Recovery is a full cold reload, not a resume.

EVICT.2MIN

Spot & preemption risk

Cheap, burstable capacity evicts in about two minutes. Serving can't risk it without a fast restore.

↳ Inference is now the dominant GPU workload: the training-to-inference ratio has flipped from roughly 70:30 to the reverse. The warm-pool tax is your fastest-growing cost line, not a rounding error.

Why cedana · what your platform gains

Pay for the traffic you serve, not the warm pool you hold.

Six benefits, each one earned by a specific piece of the architecture, not a slogan.

30s single GPU · 80s 8x GPU

Cold start to a hardware floor

cold start vs model sizebounded floor

30s / 80s floorVRAM bandwidth, not params

on-demand restore · reclaim idle replicas

Warm pools shrink to fit demand

warm pool · vs demandshrinking…

provision for averagescale for peak

predictable, bounded restore time

Autoscaling actually works on inference

replicas · track demandautoscaling…

scale up in secondsno peak-of-peak floor

cold start bounded by bandwidth, not parameters

Model-size growth stops compounding

restore time · by modelbounded floor

30s to 80s, never minutesgrowth stops compounding

coordinated checkpoint (NCCL/RDMA)

Distributed multi-GPU serving covered

8× H100 · coordinatedserving…

model-parallel servingNCCL / RDMA coordinated

transparent at the OS level · no code change

Nothing changes for your serving stack

serving · unchangedtyping…

0 code changesvLLM · SGLang · Triton

↳ One architectural move sits under all six: transparent OS-level capture of the full workload state (CUDA context and KV cache), restored on demand. Cold start becomes a function of VRAM bandwidth, not model size, up to 9.6× faster on real models at a leading inference platform.

How it works · save → migrate → resume

Your render survives the GPU underneath it.

Cedana runs at the OS and kernel level and works transparently under vLLM, SGLang, and Triton, keeping every job moving across the GPUs underneath it.

Running

A generation job lives in GPU memory.

Save

Cedana snapshots full GPU state, below the app.

Migrate

A fault or spot eviction triggers a live migration.

Resume

It resumes at the same frame. Nothing lost.

A 30-second floor on a single GPU, 80 seconds on eight. Your serving team never writes checkpoint code.

Runs on your stack · your serving framework, your scheduler

Drops in where your inference already runs.

Nvidia Dynamo●vLLM●SGLang●Triton●Kubernetes●Ray●AWS●Google Cloud●Nvidia Dynamo●vLLM●SGLang●Triton●Kubernetes●Ray●AWS●Google Cloud●

FAQ · what platform teams ask

Questions we hear first.

01We've built our own warm-pool optimization. Why Cedana?

Most platforms have. None scales across the model catalog as model sizes grow. Cedana captures at the OS level, not the application or model layer, so the whole catalog is covered and your team stops maintaining caching code.

02What's the steady-state overhead during inference?

Low single-digit percent on representative workloads. We show live overhead on the call.

03Will it work with our autoscaler and orchestrator?

Yes. Cedana works with Slurm, Kubernetes, Kueue, Ray, Armada, and Nvidia Dynamo. Cedana plus Dynamo is the marketed product form factor for inference at scale. See the docs →

04How does it handle our serving framework (vLLM, SGLang, Triton)?

Cedana captures at the OS level, so the serving framework is irrelevant to the checkpoint layer. We show resume across vLLM and SGLang on the call.

05What about compliance and data handling at our scale?

Cedana operates on GPU memory state inside your existing security boundary. It is SOC 2 compliant, ships on-premise, and supports fully air-gapped deployments.

06Does it handle distributed multi-GPU inference?

Yes. Distributed multi-GPU coordinated checkpoint is the differentiating capability. We show an 8-GPU model-parallel job restored as one unit on the call.

~ / cedana / deploy● ready

Command your
compute.

Your compute, liquid — checkpoint, migrate, and resume live GPU jobs across the fleet.

Schedule a demo Read the docs

deployK8s helm chart · SLURM plug-in

first migration<30 min

code changes0

workloadstraining · inference · HPC

AWS

Google Cloud

NVIDIA

K8s

SLURM

Nvidia Dynamo

Cold starts in seconds,not minutes.

The GPU runs behind your product, ranked by how many of you run them.

Recommendations & feed ranking

Six ways cold start inflates the bill.

The warm-replica tax

Cold start scales with model size

Autoscaling can't keep up

Diurnal & event traffic idle

Failure blast radius

Spot & preemption risk

Pay for the traffic you serve, not the warm pool you hold.

Cold start to a hardware floor

Warm pools shrink to fit demand

Autoscaling actually works on inference

Model-size growth stops compounding

Distributed multi-GPU serving covered

Nothing changes for your serving stack

Your render survives the GPU underneath it.

Running

Save

Migrate

Resume

Drops in where your inference already runs.

Questions we hear first.

Command yourcompute.

Cold starts in seconds,
not minutes.

Command your
compute.