Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
**kimi-k3-in-c** is a from-scratch inference engine for Moonshot's 2.78-trillion-parameter Kimi K3, written in portable C99 with **no BLAS, no framework, and no GPU**. The headline measurement is that the model generates on a machine with **8.24 GB of peak resident memory**, from a checkpoint that occupies **1.56 TB on disk**. The entire engine is about **176 KB** of code. The design rests on deciding where each byte lives rather than on shrinking the model. K3 is a mixture-of-experts model, and the ~1.45 TB of routed experts are never made resident: they are read from storage and multiplied straight out of their packed **4-bit MXFP4** form, with 16 of 896 experts selected per token. The dense trunk — 93 layers, packed so each is one read — stays in memory to whatever depth the operator chooses, which turns the memory floor into a dial rather than a hard requirement. An **LRU cache** holds recently used experts, sized from a recorded routing trace instead of guesswork. The project is explicit that more memory buys speed, not different answers. On the author's 124-core test machine with a fast NVMe drive, a token takes about **26.5 s at 8 GB**, 24.2 s at 32 GB, 19.8 s at 64 GB, and **5.6 s at 128 GB+**, where the model finally fits in RAM and the disk wait disappears. Output is reported as **byte-identical across every memory budget**, which is the invariant the validation suite is built around: a tiny oracle model first, then gate checks against the full checkpoint, then sustained generation. Two attention reductions carry the rest of the load. **KDA** gives an attention path whose memory does not grow with sequence length, and **MLA** compresses ninety-six heads into a single latent. Release **v1.0.0 (2026-08-07)** reports roughly 8x less arithmetic per token than the initial version, a 3.9x faster follow-up turn in a chat, and about half the cost on long prompts. The practical caveats are stated up front by the project itself. The gate is storage — you need the 1.56 TB checkpoint locally — the supported platform is **Linux x86-64**, and the weights are the base model with no chat template, so the demo outputs are continuations rather than replies. At tens of seconds per token this is a demonstration of where the memory floor actually is, not a serving stack. Licensed **Apache-2.0**.