August 25, 2026 — NVIDIA published Shadow Engine Recovery for Dynamo, its Kubernetes-native LLM inference framework: when one GLM-5.2 worker dies, a hot standby on the same GPU resumes serving in 7.3 seconds versus 283 seconds for a cold restart — roughly 39× faster recovery in a deliberate fault injection test on B200 nodes.
If you self-host frontier-class models for agent APIs, the bottleneck is rarely peak tokens/sec. It is what happens when a worker OOMs and your users stare at a 24-second time-to-first-token for five minutes.
TL;DR — what people are asking
| Question | Answer |
|---|---|
| Feature? | Shadow Engine Recovery in NVIDIA Dynamo |
| Test model? | GLM-5.2 two-worker deployment |
| Failover time? | 7.3 s vs 283 s cold restart |
| Speedup? | ~39× |
| TTFT after fault? | ~1.3 s p50 vs ~23.8 s |
| Decode after fault? | ~46 vs ~12 tok/s/user p50 |
| Mechanism? | GPU Memory Service weight sharing |
| Status? | Preview — rolling out incrementally |
How Shadow Engine works
NVIDIA's technical blog describes:
- Active worker serves traffic on a GPU.
- Shadow worker stays fully initialized on the same GPU, sharing weights via GPU Memory Service (GMS) — no second full HBM copy.
- On process failure, detection (~1.7s in test) + promotion (~5.6s) → shadow becomes active.
- Remaining cluster capacity absorbs load without five-minute reload of weights, CUDA graphs, and kernels.
Cold restart path reloads everything — hence 283s in their GLM-5.2 benchmark.
Shadow Engine vs Dynamo Snapshot
| Feature | Problem solved | NVIDIA claim |
|---|---|---|
| Shadow Engine | Runtime worker crash | 7.3s promotion |
| Dynamo Snapshot | Slow cold boot | Up to 21× faster restore (CRIU) |
| KV-aware routing | Cache reuse / prefill-decode split | Steady-state efficiency |
Use Shadow for HA; Snapshot for scale-out and pod scheduling.
What this means for what you build or pay
Agent SLOs: Coding agents and chat APIs treat more than 5s TTFT as outage. NVIDIA's table showed 201/399 requests over 5s TTFT on cold restart vs 1/398 with shadow — that is user-visible.
GPU economics: Shadow reserves capacity on the same GPU — you trade headroom for resilience. Not free HA; cheaper than doubling GPU count for active-active full copies if GMS sharing works on your model sharding layout.
Stack fit: Dynamo targets vLLM / SGLang on Kubernetes — align with how to start a small data center assumptions if you outgrow single-node DGX Spark setups.
When to plan for preview
Adopt tracking if:
- You run multi-worker LLM pools behind production SLAs
- Failures are process-level (OOM, CUDA error) not whole-node loss
- You already evaluate Dynamo for prefill/decode disaggregation
Ignore for now if:
- Single Ollama on a laptop
- You rely on managed APIs (OpenRouter) — provider owns failover
Honest limitations
- Preview — API and support may change; not default in all Dynamo installs.
- One benchmark configuration — GLM-5.2 on B200; your model size/sharding may differ.
- Same-GPU shadow does not survive full GPU hardware fault — still need multi-node redundancy.
- Memory overhead exists — shadow worker is not zero-cost; measure HBM headroom.
- GLM-5.2 naming reflects NVIDIA's test deployment — verify against your model serving stack.
Related on explainx.ai
- How to start a small data center 2026
- NVIDIA DGX Spark local LLM setup
- OpenRouter model routing for cost
- ATT AI coding costs and routing
- AI chip architectures guide
- NVIDIA Groq 3 LPX tokens
- FreeToken 753B local MoE hardware
- LangSmith Engine 2× agent debugging
NVIDIA Dynamo preview features and benchmark tables per the Aug 25, 2026 technical blog — confirm availability in your Dynamo build before promising SLA numbers to customers.