Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
KTransformers is a flexible Python-centric framework for heterogeneous CPU-GPU LLM inference and fine-tuning optimization, enabling consumer-grade hardware to run ultra-large models like DeepSeek-R1-671B. It uses Intel AMX/AVX acceleration, NUMA-aware Mixture-of-Experts placement, and multi-GPU coordination to dramatically reduce deployment barriers, achieving fine-tuning of 671B parameter models with just 70GB GPU memory plus 1.3TB RAM. The framework supports INT4/INT8 quantization, prefix caching, and a wide range of leading models including DeepSeek, Qwen3, Kimi-K2, and GLM series.
ollama
The simplest way to run LLMs locally with 165K+ GitHub stars. One-command deployment, 100+ models, REST API, and multi-platform support.
ggml-org
Pure C/C++ LLM inference engine supporting CPUs, Apple Silicon, CUDA, and Vulkan
sgl-project
High-performance LLM and multimodal model serving framework with RadixAttention and structured generation.
mlc-ai
Universal LLM deployment engine using ML compilation for cloud, mobile, and web