Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
WARP (formerly WASTE, Weight-Aware Runtime and Paging) is a dependency-free C inference engine built to run frontier mixture-of-experts models on hardware that could never hold their full weights in RAM. Rather than simply streaming layers off disk, WARP keeps a model's shared trunk resident in memory and treats the rest of available RAM as a bounded expert cache: a lookahead router predicts which experts the next layer will need and starts reading them early, while the real router still makes the final decision, so the technique changes timing rather than output. Experts use 3-bit residual vector quantization (VQ3R) while more sensitive shared weights stay at 4 or 8 bits. On a 64 GB MacBook Pro, the project's flagship demonstration is the full, undistilled 2.78-trillion-parameter Kimi K3 (982 GB container, a 29.19 GB RAM floor) at roughly 0.6 tokens per second, while the smaller 313-billion-parameter GLM-5.3-Flash needs only a 5.14 GB floor and reaches about 3.3-3.9 tokens per second on the same hardware, including image input for both models. Since version 0.6.8, WARP also converts and runs the DeepSeek-V3 family (V3, R1, and Kimi K2), though the maintainers note they have not measured throughput for that family, only correctness, via a contributed conversion checked against a reference implementation. A more experimental feature lets expert banks be striped across multiple drives via WASTE_BANK_SHARDS, but the README is explicit that no speedup from this has actually been measured yet. Validation is unusually candid: because top-k routing can make a tiny numerical difference flip which expert is used, WARP's test suite classifies routing divergences as identical, tie, or diverged rather than relying only on a numeric threshold, and its docs/LEARNED.md log records failed experiments alongside working ones. CUDA and Metal acceleration are listed as unexplored future directions, not shipped backends. A fully converted K3 container is distributable over BitTorrent so users can skip the 1.42 TB source download and the multi-hour local conversion. The code itself is written entirely by LLMs under human direction.