Cursor Open-Sources Mixture-of-Kittens: An MoE Megakernel for NVL72s

Cursor just open-sourced the kernel doing the heavy lifting behind Composer, its agentic coding model — and the headline number is that mixture-of-experts communication, not compute, was the actual bottleneck.

On August 4, 2026, Cursor Research released Mixture-of-Kittens (MoK), a production MoE training megakernel built specifically for GB300 NVL72 racks. The pitch: fuse every piece of MoE dispatch, expert computation, and combine into one fully deterministic kernel, instead of treating communication and compute as separate stages glued together by the training framework. Cursor says MoE layers had been consuming more than half of end-to-end Composer training time — and MoK now powers Composer training across tens of thousands of GPUs.

TL;DR

Why communication, not compute, was the bottleneck

Cursor's starting point is a common story in large-scale MoE training that most teams don't get to see numbers on: they'd already written custom MXFP8 and NVFP4 kernels and a "warp decode" approach for MoE inference, but those only sped up the compute half of the layer. In production, inter-GPU communication had become the limiting factor — sending tokens to the GPUs holding their assigned experts, and sending results back, was taking as long as the actual expert computation.

Two things about Cursor's GB300 NVL72 hardware changed the shape of this problem. First, an NVL72 puts 72 GPUs inside a single NVLink domain — a multi-node rack that behaves like one fast fabric, which makes fine-grained overlap of computation and communication across all 72 GPUs actually feasible. Second, the integrated Grace CPUs on GB300 (the "G" in GB300) are comparatively slow next to the GPUs — Cursor found GPU streams routinely catching up to CPU-side work and sitting idle, which forced them to minimize CPU-GPU synchronization almost entirely.

That combination — a fast enough fabric to overlap aggressively, plus a CPU too slow to lean on — is why MoK is built from scratch rather than layered on top of existing tools like DeepEP or Megatron's HybridEP.

How MoK actually works

Pull-based dispatch, push-based combine

MoE token routing can move data either by pushing (the source GPU writes tokens directly into a remote GPU's memory) or pulling (the destination GPU reads tokens from remote sources). Most existing tools default to push everywhere, on the assumption it saturates NVLink better with less protocol overhead.

Cursor's benchmarks say that assumption doesn't hold uniformly. Pull-based dispatch delivered up to 29% higher NVLink bandwidth utilization than push-based dispatch under expert imbalance, and pull-based signalling — where a rank simply loads data and starts using it once it arrives, with no cross-GPU completion signal required — cut signalling latency roughly 5.8x (18µs vs 103µs) compared to push-based dispatch, where a rank has to wait on signals from up to 71 peers and flush memory across the rack.

So MoK mixes directions deliberately: pull-based dispatch, push-based combine in the forward pass, and the reverse in backward — reusing the same routing schedule for all four operations, computed once on-device with zero CPU involvement.

Minibatches, macrobatches, and no CPU-GPU sync

MoK overlaps computation and communication at a tunable minibatch granularity — small enough to keep the tensor cores continuously fed, large enough that each expert-grouped GEMM gets at least two full "waves" of work across the GPU's streaming multiprocessors. Cursor derives a concrete sizing heuristic from the model's hidden dimension, intermediate dimension, and SM count, and validates it against microbenchmarks on Kimi 2.5-shaped MoE layers.

The harder problem is that MoE routing is dynamic — you don't know in advance how many tokens will land on a given GPU. The two conventional fixes are dropping overflow tokens (bad for quality) or synchronizing with the CPU to allocate exact buffers (bad for GB300's slow Grace CPUs). MoK's answer is a fixed-size ring buffer ("macrobatch") that cycles through memory instead of allocating a large, mostly-empty buffer up front — with dispatch and combine interleaved across the ring boundary so a buffer slot is reused the moment it's drained.

One kernel, not a pipeline of kernels

MoK is built as a megakernel — a single kernel where different SMs are software-partitioned into "comp" SMs (running expert FFNs) and "comms" SMs (running dispatch/combine), signalling each other through local counters rather than separate kernel launches. Cursor says this avoids both the launch-boundary overhead of a multi-kernel pipeline and the unreliable SM partitioning they found when trying to use CUDA streams with green contexts instead.

It also uses Cluster Launch Control (CLC), a Blackwell-native work-stealing scheduling feature, so the megakernel can yield to higher-priority inter-rack communication (like FSDP all-gather over InfiniBand/RoCE) without serializing behind it — a real concern once you're coordinating multiple NVL72 racks, not just one.

The numbers

Cursor benchmarked MoK against four public baselines — NCCL+PyTorch, DeepEP+PyTorch, DeepEP+TransformerEngine, and HybridEP+Megatron (Nvidia's own recommended option over DeepEP+Megatron on NVL72s) — across the MoE layer shapes of four widely used open-weight models: Kimi K2.7, GLM-5.2, Qwen3.5-397B-A17B, and DeepSeek-V4-Pro.

The gap between the layer-level numbers (up to 2.37x) and the end-to-end production number (1.41x) is the honest part of this release — MoE is one layer in a full training step, so a 2x-class kernel speedup on that layer alone doesn't translate one-to-one into overall throughput. Still, a 41% tokens-per-second improvement on infrastructure already running at scale is a real, not marginal, systems win.

What people are asking

Does this only help Cursor, or is it useful outside Composer training? MoK targets the DeepSeek-V3-style MoE architecture — one shared expert plus many routed experts, selected by top-k routing — that's now common across open-weight models: GLM, Qwen, Kimi (through K2.7), and DeepSeek itself. Any team training a model with that architecture on GB300 NVL72 hardware is the direct target audience; it's not useful if you're on older hardware without the NVLink domain size or Blackwell's CLC feature.

Is this a training-only kernel, or does it help inference too? MoK is explicitly a training megakernel — it computes both forward and backward passes, including router weight gradients via a SonicMoE-style calculation fused into the SwiGLU backward. Cursor's inference-side MoE work (the "warp decode" approach) is a separate, earlier project.

How does determinism affect training speed? Cursor doesn't report a determinism tax in their benchmarks — the fixed floating-point operation ordering is designed in from the start rather than bolted on, which is part of why they rebuilt from first principles instead of patching existing kernels.

Why open-source a kernel this specific to their own infrastructure? Cursor frames it as lowering the barrier to MoE research generally, and explicitly built MoK to be modifiable — including "with the help of agents" — so other labs can adapt it to different hardware. That's consistent with their broader argument in the post: AI-written kernels have gotten good enough that hand-optimized single-operator kernels are increasingly automated, freeing teams to spend effort on harder, more novel systems problems like this one.

MoK in context: the kernel-optimization landscape

MoK arrives alongside intensifying competition in efficient MoE training and post-training automation — a trend explainx.ai has been tracking across the current generation of open-weight models. Intology's Locus automated post-training pipelines to beat human-tuned Qwen3 releases; TencentDB's Agent Memory v2 is solving a different infrastructure bottleneck (shared agent memory) with a similar "build the primitive from scratch" approach. Cursor's own closing note in the MoK post — that agents now one-shot state-of-the-art single-operator kernels, freeing small teams to tackle harder distributed-systems problems — echoes a pattern showing up across AI-driven de-skilling debates and cognitive-debt discussions: the routine work is automating, and what's left is exactly this kind of first-principles systems engineering.

For teams comparing coding agents generally, Cursor's Composer sits in the same competitive set explainx.ai covers in Codex vs Claude Code — MoK is effectively Cursor showing its infrastructure work for training that underlying model.

The takeaway

Mixture-of-Kittens is a systems paper disguised as a product announcement: the actual news is that at NVL72 scale, MoE communication — not FLOPs — is where the time goes, and fixing that required rebuilding the layer as one deterministic megakernel rather than optimizing compute and communication separately. The 2.37x layer-level number is the eye-catching one; the 41% production number is the one that matters, because it's measured against Cursor's own prior DeepEP-based stack running real Composer training, not a synthetic benchmark.

Related on explainx.ai:

• Intology's Locus: Automated Post-Training Beats Qwen3
• TencentDB Agent Memory v2: Team Hub
• Codex vs Claude Code: AI Agent Comparison
• AI-Driven De-Skilling of Developers
• Cognitive Debt: What Happens When You Stop Retyping LLM Code
• DeepSeek Flash: 8 Trillion Tokens a Day on OpenCode

Official: Mixture-of-Kittens technical blog post · GitHub repository

Benchmark numbers, hardware details, and version specifics reflect Cursor's August 4, 2026 release and may change as the project evolves.