Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
High-performance serving framework for large language models featuring RadixAttention prefix caching, zero-overhead CPU scheduler, and prefill-decode disaggregation — powering trillions of tokens daily in production.
ollama
The simplest way to run LLMs locally with 165K+ GitHub stars. One-command deployment, 100+ models, REST API, and multi-platform support.
ggml-org
Pure C/C++ LLM inference engine supporting CPUs, Apple Silicon, CUDA, and Vulkan
sgl-project
High-performance LLM and multimodal model serving framework with RadixAttention and structured generation.
mlc-ai
Universal LLM deployment engine using ML compilation for cloud, mobile, and web