Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
TensorRT-LLM is NVIDIA's open-source library for optimizing Large Language Model inference on NVIDIA GPUs. It provides an easy-to-use Python API to define LLMs and applies state-of-the-art optimizations including custom attention kernels, inflight batching, paged KV caching, and quantization techniques such as FP8, FP4, INT4 AWQ, and INT8 SmoothQuant. Built on top of TensorRT and PyTorch, it delivers industry-leading throughput and latency for production LLM serving with support for MoE models and Blackwell GPU architectures.
ollama
The simplest way to run LLMs locally with 165K+ GitHub stars. One-command deployment, 100+ models, REST API, and multi-platform support.
ggml-org
Pure C/C++ LLM inference engine supporting CPUs, Apple Silicon, CUDA, and Vulkan
unslothai
2x faster LLM fine-tuning with 70% less VRAM via custom Triton kernels. Supports Llama, Qwen, DeepSeek, Gemma, and 500+ models.
SGLang (Structured Generation Language) is a high-throughput, low-latency inference engine for large language models and multimodal models, developed by the LMSYS team. With 26,600 GitHub stars and over 12,000 commits, it has become the de facto o…