Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Mixture-of-Kittens (MoK) is a training megakernel for mixture-of-experts (MoE) models, built by Cursor Research specifically for NVIDIA's Blackwell-generation NVL72 racks (GB200 and GB300, SM100/SM103 GPUs). Rather than treating expert computation and the inter-GPU communication that dispatches and combines tokens across ranks as separate steps, MoK fuses both into a single deterministic CUDA kernel that overlaps compute with networking at a configurable granularity and eliminates CPU-GPU synchronization entirely. It supports both BF16 and MXFP8 precision, covers the full forward and backward pass, and, per the README, already powers production training of Composer, Cursor's own model, rather than being a research prototype. The repository exposes two API layers: a low-level "ops" layer for calling the CUDA kernels directly with full manual control over data placement, and a higher-level "functional" layer that manages scratch memory and kernel-launch coordination, recommended for production use. A single call to schedule() builds a dispatch/combine plan reused for both forward() and backward(); MXFP8 mode expects weights prequantized ahead of time, since keeping quantization separate from the kernel leaves more room for techniques like FSDP, and a recompute_forward_context helper lets callers discard forward intermediates and rebuild them just before backward instead of holding them throughout. Tuning MoK means setting five hyperparameters, communication SM counts, a minibatch size controlling overlap granularity, a macrobatch size sized to the GPU's token ring buffer, and a schedule-capacity multiplier trading scheduling overhead against tolerance for routing imbalance, all of which the README says should be swept per workload. On standalone MoE-layer benchmarks against the fastest baseline tested, MoK measured up to 2.37x faster MXFP8 forward passes, 1.78x faster MXFP8 backward, 1.92x faster BF16 forward, and 1.58x faster BF16 backward; full methodology and end-to-end results are published in Cursor's blog post rather than the README. The project requires PyTorch 2.10+ built against a matching CUDA 13.0+ toolkit, and is released under the Apache 2.0 license.