NVIDIA Snapshot: GPU Checkpoint and Restore for Kubernetes Inference

NVIDIA open-sourced Snapshot on November 10, 2026 as a Kubernetes checkpoint system for fully initialized GPU inference. When autoscaling adds replicas or a pod dies, cold starts can take minutes on large MoE stacks. Snapshot captures CPU plus GPU state after weights load and CUDA graphs capture, then restores serving capacity in seconds on matching hardware.

For platform teams running vLLM, SGLang, or TensorRT-LLM on DSX clusters, this is the companion to Agent Substrate (agent sandboxes) and Dynamo routing: Substrate multiplexes idle agents; Snapshot multiplexes warm GPU inference.

TL;DR

Why cold starts hurt AI factories

NVIDIA's post separates cold start into startup, model load, initialization, and compilation or graph capture. On an eight-GPU MoE, those phases can sum to 6–19 minutes in the published tables while restore stays under 43 seconds.

That gap is not vanity metrics for KubeCon slides. It is lost revenue during demand spikes and SLO breaches when Kubernetes churns pods during deploys or node drains. Our semantic caching guide tackles repeat queries; Snapshot tackles repeat process state when the replica itself is new.

NVIDIA Snapshot cold-start sequence for GPU inference before checkpointing initialized replicas

Image: NVIDIA

How Snapshot works on Kubernetes

Two operations define the system:

1. Checkpoint — After the workload is fully initialized, capture portable CPU and GPU memory into an artifact.
2. Restore — Start a pod from that artifact on a node with the same GPU model and driver, skipping weight load and graph capture.

NVIDIA documents Kubernetes flows:

• PodSnapshot — checkpoint a running pod
• SnapshotJob — bring up a pod from template and checkpoint when ready
• Restore — set nvidia.com/restore-from on a new pod to name the snapshot

Workloads need a small handshake to pause before checkpoint and resume after restore. The repo ships examples for vLLM, SGLang, and TensorRT LLM.

Multi-GPU pods checkpoint as one artifact across ranks — critical for tensor-parallel MoE configs (TEP8 in the tables).

Benchmark highlights on B200

NVIDIA tested on NVIDIA B200, driver 615.71.09, VAST NFS storage. Reported speedup is cold start divided by restore time.

Representative rows from Table 1 in the post:

Key claims NVIDIA emphasizes:

• Checkpoint plus first restore beat a single cold start in every tested config
• Checkpoint creation took 15 s to 5.5 min depending on size — amortize per model revision
• On large MoE restores, CUDA restore (18–29 s) dominates CRIU CPU restore (4–11 s)

Design partners cited in the post include Crusoe (GLM-5.3 readiness ~8 min → 31 s on HGX B300) and Saturn Cloud (vLLM 2–8.5 min → 6–34 s on B300).

NVIDIA Snapshot vLLM cold start versus restore times across model sizes on B200 GPUs

Image: NVIDIA

What Snapshot does not solve

NVIDIA is explicit about boundaries:

• Image pulls and container runtime setup stay on the critical path outside measured times
• KV cache is released before checkpoint — not part of the artifact
• Node compatibility requires matching GPU model and driver
• Storage backend latency affects CRIU restore; NFS choice matters

Snapshot is not a replacement for KAI Scheduler fractional GPUs or CUDA 13.4 memory sharing. It is state reuse for inference replicas.

How to try it this week

NVIDIA's next steps on the post:

1. Install from the NVIDIA Snapshot repository linked on the blog
2. Follow the Dynamo user guide to restore workers with Snapshot
3. Join NVIDIA Dynamo Slack channel #snapshot-dev for integrations

KubeCon booth #170 is listed for live demos the week of the announcement.

If you already run Dynamo with Agent Substrate, treat Snapshot as the GPU side of fast recovery while Substrate parks CPU agent actors waiting on model replies.

Comparison with generic checkpoint tools

Linux CRIU handles process memory; NVIDIA adds CUDA restore to repopulate GPU tensors and graphs. That pairing is what separates Snapshot from checkpointing CPU-only services.

For MoE at DeepSeek V4.1 Flash scale routing, the TEP8 rows are the ones to replicate in your cluster — not the 0.6B smoke tests.

Cold-start phase breakdown (why restore wins)

NVIDIA Table 2 in the primary post assigns seconds to startup, model load, initialization, and compilation or graph capture for each configuration. On DeepSeek-V4.1-Flash vLLM TEP8, cold start totals 1110.65 s with 784.54 s in compilation and graph capture alone — the phase Snapshot eliminates on restore.

Restore Table 3 splits time across agent setup, CRIU restore, CUDA restore, and wake/remap. Even on the largest configs, CUDA restore stays near 20–29 s, showing GPU state dominates large restores but still beats minutes of torch.compile and graph capture.

Platform engineers should log the same four cold-start phases in their own stacks before trusting vendor tables. If your cluster spends only 30% of cold start on compilation, Snapshot savings shrink accordingly.

Operating checklist for platform teams

Integrating with Dynamo and KAI Scheduler

NVIDIA positions Snapshot beside Dynamo workers: restore initialized replicas so routing sees warm workers faster. That pairs with Dynamo's KV-aware decisions documented in our Dynamo recipes post.

KAI Scheduler still decides which GPU gets the pod; Snapshot decides how fast the pod becomes ready once scheduled. Do not confuse GPU fractioning with skipping compilation — they solve different bottlenecks.

Storage and security considerations

Checkpoint artifacts contain weights in GPU memory layouts and compiled kernels tied to a node class. Treat snapshots like sensitive model state:

• Restrict object-store ACLs on artifact buckets
• Rotate snapshots when publishing new quantization or engine versions
• Audit restores in regulated environments — confidential weights may persist in files longer than ephemeral pods

These points extend NVIDIA's exclusion of image pulls: supply-chain controls still matter.

Failure modes to test in staging

NVIDIA's blog does not list application-level failures after restore. Teams should deliberately test:

• Tokenizer or vocab drift between checkpoint and new image tag
• LoRA adapter swaps without recheckpointing
• Partial node failures mid-restore
• Driver upgrades that invalidate CUDA graphs

Document rollback to cold start when restore fails health checks — Snapshot should accelerate happy paths, not block safe deploys.

Economics sketch (not vendor pricing)

Use a simple model: if cold start costs $X in lost queries per minute and restore saves Y minutes per replica event, multiply by expected surge events per week. Compare against checkpoint storage GB per model revision and checkpoint CPU time (15 s – 5.5 min in NVIDIA tables).

For MoE fleets, one successful checkpoint can amortize across dozens of HPA events the same day — the blog's claim that checkpoint plus first restore beats one cold start is the breakeven inequality to plug your own traffic stats into.

Version pinning table for reproducible restores

Snapshot does not absolve teams of immutable infrastructure discipline. Pin pools per model family where restores happen daily.

What people are asking

Does this work with multi-node tensor parallel across nodes? NVIDIA's post highlights single-node multi-GPU artifacts. Cross-node restore is outside the published scope.

Is it production-ready? Open-source release with design-partner numbers; run your own soak tests on your NFS or block storage.

Relation to Run:ai going open source? Same DSX narrative — see Run:ai KAI Scheduler coverage. Snapshot is another DSX-aligned component.

Related reading

• NVIDIA Agent Substrate on Kubernetes
• vLLM versus SGLang comparison
• Semantic caching for LLM apps
• LowRAM KV cache compression research
• CoreWeave Blackwell training deals
• NVIDIA Korea sovereign AI forum

NVIDIA version numbers and timings are taken from the November 10, 2026 technical blog; validate on your hardware before production rollout.