SGLang-Omni is the SGLang project's answer to a problem that text-only inference stacks handle badly: speech and omni models do not decode in one uniform loop. A text-to-speech request passes through preprocessing, an encoder, an autoregressive engine, a talker, a decoder, and a vocoder — stages with different compute patterns, different dependency structures, and different resource needs. Serving that as a single monolithic forward pass wastes hardware. The project is Apache-2.0, published by the sgl-project organization, and has reached 924 stars and 370 forks since going public on 7 January 2026. The current release is v0.1.3, published on 20 August 2026 and installable from PyPI.

The design decision that defines the runtime is multi-stage decoding. SGLang-Omni owns the pipeline topology, stage lifecycle, inter-stage transport, and model-family integration layer, while composing with the main SGLang engine for high-performance autoregressive scheduling where that applies. Each stage runs behind a scheduler matched to its workload — SGLang-backed autoregressive scheduling for the heavy generation step, lightweight schedulers for preprocessing, streaming loops for the vocoder. A control plane coordinates requests while a separate relay data plane moves tensor payloads between stages over shared memory, NCCL, NIXL, or Mooncake backends, so the transport can be matched to the deployment rather than assumed.

Model coverage is the more immediately useful part. On the speech-generation side the cookbook documents Higgs Audio v3, MOSS-TTS (both hosted and local variants), Fish Speech S2-Pro, Qwen3-TTS, Voxtral TTS, Ming-Omni-TTS, dots.tts, and ZONOS2. Transcription and diarization run through /v1/audio/transcriptions with Qwen3-ASR, Fun-ASR, ARK-ASR, and MOSS-Transcribe-Diarize — the last of which returns speaker labels and timestamps when asked for verbose_json. Omni chat covers Qwen3-Omni and Ming-Omni, taking multimodal input and returning text or audio. In August the project added day-0 support for MiniMax Music 3 (a MiniMax model, served here rather than authored here), turning lyrics plus a caption into a 32 kHz stereo song on the /v1/audio/speech endpoint.

The API surface is deliberately OpenAI-compatible, exposing multimodal chat, speech generation, batch speech, streaming speech, uploaded voices, and transcription — which means existing client code aimed at OpenAI's audio endpoints can be pointed at a self-hosted deployment without a rewrite. An accompanying SGLang-Omni Router provides a multi-worker front door handling health, readiness, lifecycle, and capability discovery, so a fleet serving different models can sit behind one address.

The caveats are worth stating plainly. Hardware support is narrow: NVIDIA CUDA is the only fully supported backend, and Intel GPU (XPU) is marked experimental, with Qwen3-ASR, Qwen3-TTS, and Qwen3-Omni serving end-to-end via PyTorch XPU. There is no CPU or Apple Silicon path here — this is server-side infrastructure. Version numbering is honest about maturity, and the v0.1.3 install line still requires --prerelease=allow. August also brought a TTS architecture refactor covering shared pipeline state, engine construction, reference encoding, capability metadata, and vocoder scheduling, which suggests the internals are still moving. Teams that want a stable audio-serving API today will find one; teams that want a frozen one should wait.