Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Naive-N0.5-Flash is an open-weight Mixture-of-Experts language model from NaiveAI, built for coding and AI research and development work. It has 309B total parameters with 15.5B active per token, 48 transformer layers, and a native context window of 1M tokens (1,048,576 in the config). The model builds on Xiaomi's open-weight MiMo-V2.5 base. Most layers use sliding-window attention with a 128-token window, and the global-attention layers are replaced by DeepSeek Sparse Attention: a lightweight indexer with 16 query heads scores the full history and the backbone attends to the top 2,048 tokens. The network has 39 sliding-window layers and 9 sparse-attention layers, with no full-attention layers, and it swaps the original MLA-based DSA for grouped-query attention with four KV groups. After the architecture change, the model went through 3.25T tokens of multi-stage training: 50B tokens of indexer warmup, 3T tokens of sparse attention training and 200B tokens of learning-rate decay. NaiveAI also describes NaiveRT, an inference system combining mega-kernel fusion, Programmatic Dependent Launch and speculative decoding, which it reports delivers 50 tokens/s per user in Standard mode and up to 2,000 tokens/s in Ultrafast mode. The card shows coding, agentic and AI R&D benchmark results only as images, with notes on how each competitor score was sourced; the evaluations use Claude Code 2.1.207 with a 1M-token context, temperature 1.0 and top-p 0.95. For deployment the model supports FP8 mixed-precision inference and requires FP8-capable NVIDIA GPUs; the weights occupy roughly 315 GB, plus additional GPU memory for inference, and the quick start uses transformers 5.17 or newer with trust_remote_code. Weights and inference code are released under the MIT license.