LoginSchedule demo
cedana / blog · field reports & engineering deep-dives

Writing from the
cedana team.

Engineering notes, benchmarks, and field reports covering how we build the automation layer for AI factories.
Featured
Blueprint schematic of an eight-GPU node accumulating session state, with a state gauge rising toward the VRAM capacity lineBlueprint schematic of an eight-GPU node accumulating session state, with a state gauge rising toward the VRAM capacity line
FeaturedEngineering

The Era of Stateful Inference: How to Improve Cost per Token.

We are entering the stateful inference era, driven by frontier models with longer context windows, longer in-flight sessions, and single instances spanning 8, 16, or more GPUs.

Updated · Cedana Editorial
read →
All posts · 98

insights from the team.

Field reports, engineering deep-dives, and benchmarks from the Cedana team.

TopicGPU utilization

Cedana. Blueprint of the same GPU card drawn three times down the left. The first is cut by six vertical walls into seven slices holding grey blocks of different heights, with a dashed empty card to its right. The second holds three overlapping translucent blocks with a red crack running through them, and a box to its right of three red dashes. The third is filled by one solid blue block, with a row of three blocks to its right, grey, blue, grey, separated by saved-state icons.
· Cedana Editorial

Sharing GPUs without fixed MIG slices

Compare MIG, time-slicing, MPS and serial sharing through checkpoints. Understand isolation, fixed slice sizes and the save-and-restore cost of switching jobs.

read →
Cedana. Blueprint of a laptop on the left joined by a grey dashed cable to a GPU card on the right whose status bar is grey. A clock sits above the cable and a red cross cuts it. Below, a blue save icon marked checkpoint sends a blue arrow into the card, where a blue block now sits.
· Cedana Editorial

Why an idle notebook keeps its GPU until you kill it

See why idle notebooks and Ray actors keep their GPUs, how shutdown tools release them, and what a checkpoint must preserve before a session ends.

read →
Cedana. Blueprint of two bars. The top bar is filled six tenths in grey, marked busy sixty percent. Two dashed guides drop from that grey portion to a second bar of the same width, split into a red hatched part marked lost, a dark part, and a blue part marked kept thirty-four percent.
· Cedana Editorial

How much of your GPU utilization was work you kept?

Distinguish GPU activity, MFU and useful output. Use job accounting and startup timings to estimate how much paid GPU time produced work you kept.

read →
Cedana. Blueprint of two rows of eight GPU cells. In the top row two cells are grey and a blue job outlined in dashes sits across three of the empty ones, with an arrow arriving from the left marked owner returns. An arrow leads down to the second row, where all eight cells are grey, and the same blue job stands whole to the right above a save icon marked checkpoint.
· Cedana Editorial

How to fill idle GPUs without killing the job that fills them

Compare how GPU schedulers lend and reclaim idle capacity, what preemption costs borrowers, and where checkpointing can preserve their work.

read →
Cedana. Blueprint of four outlined node rows of eight GPU cells, grey where in use and white where free, with two free cells scattered in each row. To the right, one eight-cell job outlined in dashed blue, marked 8-GPU job, blocked by a red cross.
· Cedana Editorial

Why a cluster with free GPUs still cannot place an 8-GPU job

See why scattered free GPUs cannot fit a large job, what bin packing and consolidation change, and why moving running state matters for defragmentation.

read →
Cedana. Blueprint of a board of eight node rectangles in two rows, most holding grey job blocks each fixed with a black push-pin where it landed. One job in the top right is a dashed blue outline carrying a saved-state icon and no pin, with a long blue dashed arrow curving from it to a solid blue block that has landed in the one empty node in the row below.
· Cedana Editorial

What would a GPU scheduler do differently if it could move running jobs?

Compare how GPU schedulers handle placement, preemption and time limits, and what checkpointing could change once a job has already started.

read →
Cedana. Blueprint of four tall worker columns each under a green tick, marked all healthy. The first is filled almost to the top in red; the other three hold a shallow blue fill. A blue dashed arc carries the first worker to a fifth, dashed column at the right, filled pale blue to the same height, and a save icon marked checkpoint sits under the first column.
· Cedana Editorial

Why a healthy inference worker can still be in the wrong place

Understand how GPU topology, decode load and hardware fit affect inference workers, and why admission-time placement cannot rebalance running sessions.

read →

Showing 7 of 7 posts