LoginSchedule demo
cedana / blog · field reports & engineering deep-dives

Writing from the
cedana team.

Engineering notes, benchmarks, and field reports covering how we build the automation layer for AI factories.
All posts · 98

insights from the team.

Field reports, engineering deep-dives, and benchmarks from the Cedana team.

TopicInference performance

Cedana. Blueprint of eleven horizontal bars stacked top to bottom, each longer than the one above, grey along almost their whole length with a short blue block at the right end of every bar. From the longest bar a grey dashed arrow runs to a small GPU card at the lower right with a black push-pin above it.
· Cedana Editorial

How long do agentic sessions run, and where does their state live?

Explore published coding-agent traces to see how turns, cache reuse and pauses shape session affinity, GPU memory use and the cost of a lost worker.

read →
Cedana. Blueprint of a large square tank outlined in ink, its lower half filled dark grey with faint horizontal lines, a dashed line across its surface, and the upper half stacked with fifteen thin blue slabs, the topmost brighter, with a row of small circles queued at its right edge and an arrow pointing in.
· Cedana Editorial

Your KV cache now rivals your weights

Calculate KV cache bytes from model configuration, precision and context length, then estimate how many live sessions fit after the weights are loaded.

read →
Cedana. Blueprint of a large outlined box holding six horizontal slots. Five slots are filled from the left with pale-blue bars of different lengths; the fifth slot is empty. Outside the box on the left, three short dark grey bars are stacked, and a blue arrow leads from the top one into the empty slot.
· Cedana Editorial

The engine layer: batching, caching, and configuration as a commitment

Understand how continuous batching, paged KV memory and prefix caching affect serving performance, and why changing a worker's regime means launching a new one.

read →
Cedana. Blueprint of two pools of four workers, marked prefill and decode. In the prefill pool each worker holds a short grey bar; grey dashed arrows cross to the decode pool, where each worker is stacked with blue slabs to a different height. From the tallest a blue dashed arrow leads right to a dashed outline holding the same slabs in pale blue, above a save icon marked checkpoint.
· Cedana Editorial

Does disaggregated serving remove the need to move GPU workers?

See why separating prefill and decode does not make serving workers stateless, and what moving a worker means for its KV cache, sessions and startup cost.

read →
Cedana. Blueprint of two GPU cards side by side, each holding three horizontal bars: a dark bar of equal length in both, a blue bar noticeably shorter on the left card than on the right, and a second blue bar shorter still on the left. Below the left card a block of six parallel lines points up into it; below the right card a row of six blue dots points up into it; a red dashed line between the two is struck through.
· Cedana Editorial

H100 for prefill, H200 for decode: the mismatch you pay for twice

Compare H100 and H200 memory and compute for prefill and decode. Account for cache-transfer cost and why moving across GPU models still requires a cold start.

read →
Cedana. Blueprint of two rows of eight outlined boxes, each box holding four small tiles. In the top row one tile per box is blue and seven small arcs hop from box to box across the whole row. In the bottom row all four tiles are blue in only the fourth and fifth boxes, joined by a single arc, and every other tile is pale grey.
· Cedana Editorial

Expert placement in a mixture-of-experts deployment is a systems decision

Separate a model router's expert choices from the deployment layout you control. Review expert placement, attention parallelism and communication overlap.

read →
Cedana. Blueprint of four towers standing on one shared foundation, each tower up to three tiers tall. The first has one solid grey tier and two dashed above it; the second one solid tier with a small red cross where the next would be; the third two dotted outlines; the fourth two solid blue tiers and a dashed blue one on top.
· Cedana Editorial

What NVIDIA Dynamo Snapshot and Modal GPU memory snapshots restore today

Compare Dynamo Snapshot, Modal, InferX and Cedana by captured state, restore compatibility, supported workloads and the limits behind their benchmarks.

read →
Cedana. Blueprint of an LLM worker's cold start as a staircase descending in seven treads to a ready floor, the second tread far longer than the rest, marked nine to thirty-four minutes. One lane below on the same span, a short blue bar marked fifty-seven to seventy seconds.
· Cedana Editorial

Where the minutes go when an LLM worker cold starts

Trace LLM cold starts through weight loading, compilation and warmup. Compare caching fixes with checkpoint restore and understand the storage limits.

read →
Cedana. Blueprint of a demand curve rising and falling above a baseline, with a thick dashed floor line drawn high across it; the area between the curve's troughs and the floor is hatched, capacity paid for and idle, while the peaks rise above the line. At one trough a small blue block sits on the baseline under a saved-state icon, the width of a restore.
· Cedana Editorial

Can LLM inference scale to zero without paying for warm replicas?

Understand when LLM inference can scale to zero, what warm replicas cost, and how restore time and request latency determine the capacity you keep ready.

read →
Cedana. Blueprint of one GPU card at the centre with a blue model block resident on it, and a dashed arc above it carrying two saved-state artifacts, the parked models. A grey dashed arrow leaves the card for the left artifact and a thick blue arrow comes in from the right one. A thin blue dashed line also runs from that artifact to a second, dashed GPU card at the lower right.
· Cedana Editorial

How to swap models on one GPU without a cold start

Compare GPU model swaps using vLLM sleep mode, SGLang and checkpoint restore. Learn where parked state lives and what each swap still costs.

read →

Showing 13–22 of 22 posts