Mixture-of-Kittens: our open-source MoE megakernel for NVL72s
Cursor open-sourced a deterministic MoE training megakernel for GB300 NVL72 racks. It fuses communication with compute and reports 1.41× end-to-end training throughput in production.
Cursor open-sourced **Mixture-of-Kittens**, a deterministic MoE training megakernel for GB300 NVL72 systems. It reports up to **2.37× MXFP8 forward throughput** versus the fastest public baseline and **1.41× end-to-end tokens per second** in production.
Teams training MoE coding models should profile communication as part of the layer, not optimize matrix compute alone. MoK uses pull dispatch, push combine, a reusable device-side schedule, and no CPU-GPU synchronization in that path.
Cursor open-sourced **Mixture-of-Kittens**, a deterministic MoE training megakernel for GB300 NVL72 systems. It reports up to **2.37× MXFP8 forward throughput** versus the fastest public baseline and **1.41× end-to-end tokens per second** in production. Teams training MoE coding models should profile communication as part of the layer, not optimize matrix compute alone. MoK uses pull dispatch, push combine, a reusable device-side schedule, and no CPU-GPU synchronization in that path. The design targets DeepSeek-V3-style MoE layers and NVL72 hardware, where 72 GPUs share an NVLink domain. The supplied results come from Cursor, and the material does not establish gains on other topologies or workloads.
This moves the open-model discussion beneath model choice and fine-tuning APIs to MoE systems design: on one tightly specified NVL72 topology, communication scheduling and eliminating host synchronization materially improve training throughput. It strengthens the case for workload-level infrastructure measurement, while narrowing the evidence to DeepSeek-V3-style layers on GB300 NVL72 rather than establishing a portable gain for other models or clusters.