We increase tokens per GPU-hour at fleet-level scale. Transparent checkpoint, migrate, and resume turns GPU-hours lost to cold starts, failures, idle windows, and stranded placement into delivered tokens. Real-time GPU virtualization optimizes runtime execution to increase token throughput. These proprietary primitives deliver new capabilities for inference.
Fleet output is useful GPU-hours times tokens per hour. Cedana raises both: lost hours become delivered tokens, and working hours deliver more. Same hardware, same models, same serving stack.
Cold starts, idle windows and failures stop costing time, and each working hour delivers more.
An 8-GPU, trillion-parameter model restores from checkpoint in about 60 seconds instead of a 10 to 40 minute cold start. Autoscaling provisions workloads closer to demand, not ahead of it.
A new spot tier detects idle windows across the fleet and runs long-horizon, stateful inference through them. When the GPU is reclaimed, the job moves with its state. No preemption window required.
A node fails and the job resumes on healthy hardware. A security patch lands and workloads pause, then resume where they left off. Priorities change and jobs follow them. Fleet-wide, with no loss of state, including in-flight sessions.
Runtime execution is optimized beneath the engine for 10 to 20% more token throughput on frontier models. It stacks on top of your existing speculative decoding and kernel optimizations.
The same mechanism from one GPU to the fleet, with no changes to models, engines or orchestrators.
Scale across single GPU, multi-GPU and multi-node. Prefill and decode pools autoscale independently and rebalance in seconds.
Route each task to the right model and escalate to a frontier model only when the task needs it. Cedana swaps models in and out of VRAM in seconds increasing tasks productivity by over 2x at lower costs.
Bring your own stack, or start with a complete frontier-scale stack: NVIDIA Dynamo, vLLM and SGLang support, and Cedana Storage Express.
Same hardware, same models, native versus Cedana. Every number links to the run it came from.