» Cedana / Kubernetes / Scavenger queue

Turn idle GPUs into capacity. Automatically.

Cedana turns your cluster’s idle time into finished batch work. Long-running jobs run as low-priority pods in the gaps; when a priority pod arrives, the Kubernetes scheduler preempts, Cedana checkpoints inside the grace period, the Job is re-created, and the workload resumes where it stopped on the next free GPU, on any node. Completely automated. Priority always comes first. No scripts, no schedules, no code changes.

Results from a 7.4-hour run

Measured · not simulated

The following results are from a single continuous 7.4-hour run on an AWS EKS cluster with 8 NVIDIA A10G GPUs across two nodes. Eight scavenger workloads ran as Jobs in a low-priority PriorityClass; high-priority Jobs preempted them through the stock kube-scheduler, which normally kills the pod and loses all progress.

Here, Cedana checkpointed each workload at preemption to shared EFS and automatically restores it on idle GPU windows, often on the other node. It detects idle capacity automatically.

Priority jobs used 35% of the cluster’s GPU time, leaving 65% idle. The scavenger workloads were preempted 140 times, every one restored from its newest checkpoint, and all eight ran to completion.

Of the idle GPU time, 73% was converted to useful work (91% while jobs were waiting), with 6% spent on checkpoint and restore. Median handover from a priority pod’s submission to its first CUDA kernel on the preempted GPU was 13.2 seconds, and 71 restores migrated onto a different node.

000:00
»01 · RUN COUNTERS
Idle GPU time recovered by Cedana0 %

of 0.0 h idle GPU time so far

Work kept · with vs without Cedana
With Cedana0.0 h
Without · simulated0.0 h

Cedana saved 0.0 h

0Priority jobs run
0Preemptions
0Resumed from checkpoint
0Cross-node resumes
0Units rewound
0Restarts from zero
0Restores cut short · no loss
0 / 8Jobs finished
»01B · GPU-HOURS RECOVERED FOR EFFECTIVE WORK
0.0Cluster · 8 GPUs
0.0node-45-250 · 4 GPUs
0.0node-58-22 · 4 GPUs
»02 · GPU TIMELINE
~ / cedana / kubernetes / eks-pre-8h-5 / gpusreplay
PRIORITY JOBPRIORITY WAITING FOR GPUIDLERESTORINGCHECKPOINTING◆ CHECKPOINTED   ▶ RESUMED   ✕ CUT SHORT   ★ FINISHEDPRESS C: WITH / WITHOUT CEDANA
~ / jobs / progress:: units held
THIN GREY BAR: SIMULATED PROGRESS IF EVERY PREEMPTION RESTARTED THE JOB FROM ZERO, SAME SCHEDULE. NOT MEASURED.
~ / idle time / per gpu / wasted otherwise
PRIORITY WORKIDLE TIME RECOVERED · WASTED WITHOUT CEDANACHECKPOINT / RESTOREKEPT SO FAR · NOT YET PREEMPTEDTHROWN AWAY ON PREEMPTIONSTILL IDLE
»03 · FULL RUN
~ / eks-pre-8h-5 / 442 minclick to seek

SOURCE: pod accounting, the Cedana daemon and propagator event stream, the kube-scheduler’s Preempted events, and the job PROGRESS logs of run eks-pre-8h-5 (2026-10-27 15:18:54 UTC) · Amazon EKS v1.35.8 · 2 × g5.12xlarge · 8 × A10G · cedana-helm-0.9.406 · termination grace 120 s. Restore phase runs from pod start to “workload up”; checkpoint phase from “dump starting” to “dump completed”. Progress is each job’s own PROGRESS counter. Every preemption is paired with the priority pod the scheduler named. The “without checkpointing” bar is simulated: the same preemption times with the job restarting from zero each time.