Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Strata is a C++ inference engine and one-click installer from Niko1221 that runs Qwen3.8-Flash-Next, a 125B-parameter mixture-of-experts model, on a single consumer NVIDIA GPU plus system RAM, on Windows 10/11 or Linux. The README describes the model as 24,576 small experts with 10 active per token. The GPU holds the always-needed parts of the model, the KV cache and a cache of the most-used experts, which adapts to the conversation. System RAM holds all experts and the CPU computes the ones missing from the GPU at the same time as the GPU works, while a 28.8 GB n-gram lookup table stays on the SSD. The model's own MTP layer drafts up to 3 tokens per pass for speculative decoding, and the README says the big model still decides every word. It builds on parts of llama.cpp / ggml (MIT). Setup asks which model, size, context length and whether to enable images, then downloads roughly 70 GB and opens a browser app at http://127.0.0.1:8080 with Chat and Monitor tabs. Apps can connect through an OpenAI-compatible API at /v1 or an Anthropic Messages endpoint at /v1/messages. The README's headline requirements are one NVIDIA card (12-24 GB) and 64 GB of RAM; the install section lists an RTX 20/30/40/50 card with 12 GB of VRAM or more, and the details page says 8 GB runs, slowly. Quantized sizes are Q2_0, IQ2_XS, IQ3_XXS and IQ3_S, plus a coding-focused variant. The README's speed table was measured on one rig, an RTX 5070 (12 GB) with a Ryzen 5 7600 and 64 GB of RAM: Q2_0 writes 93 tokens/s in short chat and 74 tokens/s at 128K context, while IQ3_S writes 53 and 46 tokens/s. An RTX 3090 (24 GB) is only estimated at roughly 100-140 tokens/s. Strata is MIT-licensed; the model weights are not part of the repository and keep their own licenses.