Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Project Maya is a C++ and CUDA inference engine built for a single model, GLM-5.3-Flash from zai-org (MIT license), which the README describes as 321 B parameters with about 18 B active per token and a context of up to 1 M tokens. The engine keeps the experts a conversation uses most on the GPU, the next ones in RAM and the rest on an NVMe SSD, and moves them as the chat goes on; it measures the GPUs, CPU, RAM and SSD and tunes itself to them. It runs on one NVIDIA GPU (compute capability 7.0 or newer) or up to 16 that share the model, with AMD and Windows support marked experimental. The weights are not Maya's own: its Maya-S, Maya-S24, Maya-M, Maya-M-Derisked and Maya-L quants are made from Z.ai's FP8 release and range from 94.7 GB to 156.3 GB, and the README reports 97.7-99.2% of the FP8 model's zero-shot accuracy for the measured ones. The server offers OpenAI- and Anthropic-compatible APIs with tool calls, a browser chat and live monitor, image input, selectable thinking levels, and up to 8 requests answered together. The README's table of user-reported results includes 118 tokens/s decode on four RTX 4090 24 GB cards (Maya-S24, 62 GB RAM, Xeon w7-3445) and 30+ tokens/s on an RTX 5090 32 GB with 96 GB DDR5 (Maya-M); prefill figures are for an 8K-token prompt. An optional DFlash2 drafter by Anbeeld speeds decoding on some setups but is licensed CC BY-NC-ND 4.0, non-commercial only. Requirements include 32 GB RAM, a fast NVMe SSD, and on Linux CUDA 12.x and Python 3.10+; setup builds everything once in 20-40 minutes plus download. The engine began from the MIT-licensed Strata project and uses ggml / llama.cpp components. Project Maya is MIT licensed.