Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
**Audio8_TTS** is a **0.6B-parameter multilingual text-to-speech model with zero-shot voice cloning**, published by Audio8-AI under **Apache-2.0**. The repository opened on **29 July 2026** and has gathered **512 stars and 46 forks** in roughly two weeks, with the most recent push on 9 August. It is explicitly labelled a **Preview checkpoint** — the team states that language coverage is intentionally limited in this release and that broader multilingual and Chinese-dialect support is planned for later. The architecture is a **DualAR** design the authors credit to **Fish Audio S2 Pro**. A **slow autoregressive transformer** — 24 layers, width 896, 14 attention heads over 2 KV heads — predicts one semantic token per audio frame. A **fast AR transformer** of just 4 layers then predicts that frame's codec codebooks, conditioned on the slow branch's hidden state and the codebooks already emitted. Both branches use static KV caches during generation. The acoustic side runs **10 codebooks with 4,096 entries each** through a **44.1 kHz codec** at 2,048 samples per model frame, or about **21.5 frames per second**, with context up to 2,048 packed text/audio positions. The main model counts **601,159,424 parameters** excluding the codec, and the checkpoint **bundles its own neural codec** — so reference encoding and waveform decoding need no separate model download. The **11 recommended languages** are Cantonese, Chinese, Dutch, English, French, German, Italian, Japanese, Korean, Polish, and Spanish. Voice cloning is zero-shot: pass `--reference-audio` alongside a `--reference-text` transcript that matches what is actually spoken in the clip, and the model adopts the voice. Generation without any reference works too. A batch mode reads a JSONL manifest, writes `manifest.jsonl` and `failures.jsonl`, and skips WAVs that already exist unless `--overwrite` is passed — the kind of resumability that matters when a batch of thousands dies partway. Deployment covers both ends of the hardware range. The `onnx_runtime/` directory is a **standalone CPU path** using weight-only **INT4** slow/fast AR models with FP16 activations, KV caches, and codec — no PyTorch, no Transformers — offering CLI inference, a local web and HTTP service, streaming PCM output, and reference-voice registration. Measured on an **Apple M2**, online sessions hold about **1 GiB of memory**, and the code deliberately releases those sessions before loading the codec encoder during voice registration to keep the peak down. At the server end, `sglang_omni/` is an **OpenAI-compatible service** with SGLang paged attention and dynamic batching, installed as an independent plugin that does not overwrite SGLang Omni core files. The limitations are stated plainly rather than buried. Quality **degrades past roughly 150 characters per input**, so long passages must be split into segments by the caller. Python 3.10+ and a CUDA-capable GPU are recommended for the PyTorch path. And the SGLang adapter leans on internal SGLang Omni interfaces, so it must be deployed against the **pinned tested revision** (`68a5723`, SGLang Omni 0.1.0 with SGLang 0.5.8) rather than the latest `main` — a real operational constraint for anyone tracking upstream.