Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
TensorFold serves language models on Apple Silicon and NVIDIA GPUs through an OpenAI-compatible API, with each model family supplying its own kernels and draft verification. Both backends (MLX on a Mac, CUDA on NVIDIA) serve chat completions, completions, OpenAI Responses and Anthropic Messages endpoints, and the server listens on 127.0.0.1:8080 by default. Its decoding is exact against the same engine: a draft token is accepted only when it equals the token the engine would produce serially, and exactness does not carry across MLX versus CUDA, different quantizations, or different tensor-parallel rank counts. Supported families listed include Nemotron 3.5 Lightning, Qwen3.8-27B, Qwen3.8 Flash Next, GLM-5.3-Flash, Gemma 4 26B-A4B, DeepSeek-V4-Flash and Ternary Bonsai 2 27B, drafted through MTP heads, DFlash2 models or copies of the context; the EXL3 checkpoints for Qwen3.8 are marked experimental. GLM-5.3-Flash on MLX and DeepSeek-V4-Flash are listed for a 256 GB Mac, and GLM on CUDA requires two ranks. Optional image input is available for GLM-5.3-Flash and dense Qwen checkpoints with --vision. Python 3.11 or newer is required, plus MLX 0.32.2 or newer on a Mac. The CUDA kernels need compute capability 8.9 or newer (Ada, Hopper, Blackwell including DGX Spark and RTX 50); the RTX 40, Hopper and B200 builds are compiled and bit-checked on Blackwell but not yet run on those cards, and RTX 30 cards are not supported. Memory-class results for Macs are mostly marked TBD, and the one filled-in row comes from a community user's run on a 64 GB M5 Pro with the older 0.3.5.1 release. Install with pip from the GitHub repository or Homebrew on a Mac. The project is Apache-2.0 from version 0.6.0, while earlier releases up to 0.5.0 were MIT, and model weights keep their own licenses.