Launching each kernel in a model's forward pass individually means the CPU issues a separate launch instruction per operation, and at small batch sizes that launch overhead can outweigh the GPU compute itself. A CUDA graph captures the whole sequence of kernel launches once and replays it as one call, eliminating per-kernel CPU overhead on every subsequent request. Perplexity's embedding-serving stack (ROSE, described in their September 2026 'Fast Embeddings on GPUs' post) captures whole-model graphs for pplx-embed and uses lazy graph capture — running the first call eagerly as warmup and capturing on the second call — to spread the multi-minute capture cost across hours of serving instead of paying it at startup, trading slightly worse cold-start p99 latency for much faster fleet scaling.