During autoregressive inference, each token generation reads the entire model's weights from memory but performs relatively little computation per byte loaded, making memory bandwidth — not FLOP/s — the binding constraint. HBM3e on modern GPUs provides around 4–8 TB/s, and the ratio of bandwidth to compute determines the arithmetic intensity crossover. This is why quantization (fewer bytes per weight) speeds up inference almost linearly even though it reduces precision.