Ollama

ollama

The simplest way to run LLMs locally with 165K+ GitHub stars. One-command deployment, 100+ models, REST API, and multi-platform support.

llama.cpp

ggml-org

Pure C/C++ LLM inference engine supporting CPUs, Apple Silicon, CUDA, and Vulkan

SGLang

SGLang (Structured Generation Language) is a high-throughput, low-latency inference engine for large language models and multimodal models, developed by the LMSYS team. With 26,600 GitHub stars and over 12,000 commits, it has become the de facto o…

SGLang

sgl-project

SGLang is a high-performance open-source serving framework for LLMs with RadixAttention prefix caching, zero-overhead CPU scheduling, and prefill-decode disaggregation — deployed across 400,000+ GPUs at xAI, LinkedIn, and Cursor.