Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Splash is a local inference engine for Apple Silicon from Inco AI (incoai), released under the Apache-2.0 license (model weights it serves keep their own licenses). Installed with brew install incoai/tap/splash, it runs on an Apple M3 or newer chip with macOS 26.4 or later, Homebrew, and at least 36 GB of unified memory (48 GB or more recommended), and currently serves two model packages: incoai/Qwen3.8-27B-Splash (a 4-bit quantization of Qwen3.8-27B bundled with its own DFlash 2 speculative-decoding draft, 17.4 GB) and incoai/Qwen3.6-35B-A3B-Splash (the same treatment applied to Qwen3.6-35B-A3B, 20.9 GB); plain MLX or Transformers checkpoints do not work with the server. The server exposes OpenAI's Chat Completions and Responses endpoints and Anthropic's Messages endpoint, all with streaming, tool calls, JSON Schema output, images, and inline PDFs, plus /tokenize and /apply-template utility routes and /v1/judgments and /v1/systemone for scoring without generation. There is no configuration file: at startup Splash computes a memory budget from available unified memory and sets context length (up to the model's native 256K window, memory permitting), KV cache format (INT8 by default, BF16 optional), and batch limits automatically. Every served model gets its own fused Metal kernels compiled for its exact tensor shapes and mapped zero-copy from disk, so there is no Xcode or compiler toolchain needed on the machine running it, and speculative decoding through the model-specific DFlash 2 draft is the default decode path rather than an optional flag. On a 48 GB M5 Pro, Splash's own SPEED-Bench coding-prompt measurements (1,024-token output cap, reasoning set to medium for the 27B) report it leading the next-fastest engine it tested against at every prompt length: short-prompt decode of 210 tok/s (1.7x) for Qwen3.6-35B-A3B and 74 tok/s (2.0x) for Qwen3.8-27B, 32K-prompt prefill of 2,011 tok/s (1.3x) and 363 tok/s (1.2x), and cached time-to-first-token on a 32K replay of 123 ms (6.6x) and 282 ms (7.3x); the company reports the lead widening under load, to 3.8x at four concurrent 32K requests on the 35B. Splash also integrates with LM Studio Bionic through a documented setup guide.