Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
**MiniMax Music 3** is MiniMax's open-weights music generation model, released to GitHub on **11 August 2026** and now at **478 stars and 39 forks**. The weights went up on Hugging Face at `MiniMaxAI/MiniMax-Music3` a few days earlier, on **7 August 2026**, and have already pulled **8,639 downloads and 850 likes** — a gap worth noting, since the community found the checkpoint before the repository existed. The pitch is complete songs: conditioned on lyrics and a music description, it generates structurally coherent tracks **up to five minutes long** with expressive vocals and evolving arrangements, output as **32 kHz, 16-bit stereo WAV**. The architecture is a hierarchical autoregressive design that deliberately splits two jobs usually crammed into one model. A **Global LLM (8B)**, initialized from **Qwen3-8B**, predicts the first RVQ codebook frame by frame and carries the song's long-range semantic and structural progression. A much smaller **Local LLM (0.6B)** predicts the remaining acoustic codebooks within each frame and restores fine-grained detail. Synthesis then departs from the usual token-decoding route: instead of decoding only from discrete RVQ tokens, the module fuses the final hidden states of both LLMs and passes those continuous representations through **Flow Matching (2.4B)** into a **Flow-VAE decoder (123M)**. MiniMax says the continuous path preserves richer acoustic information for vocal articulation, instrumental texture, and temporal continuity, and that the Flow-VAE is adapted from MiniMax Speech and retrained for the dynamic range of music. The training tokenizer uses **eight layers of Residual Vector Quantization** — a first semantic codebook of **16,384** entries capturing musical semantics and structure, plus seven acoustic codebooks of **1,024** entries each for residual detail. Training optimizes the semantic codebook first, then jointly trains all eight. At inference, waveform synthesis works from the fused LLM hidden states and does not need the discrete tokenizer decoder at all. Control comes through two complementary inputs. **Lyrics** carry the words and may include explicit section tags — `[Intro]`, `[Verse]`, `[Pre-Chorus]`, `[Chorus]`, `[Post-Chorus]`, `[Bridge]`, `[Instrumental]`, `[Solo]`, `[Outro]`. A **music description** defines style, emotional progression, vocal performance, instrumentation, arrangement, and production profile. For tighter control the team recommends a **Structured Caption** with three sections — Global Metadata (genre, BPM, key, scale, emotional progression, production profile), Vocal Details (gender, timbre, performance style, harmony, effects), and Arrangement (instruments, section-level evolution, groove, bass, percussion, spatial effects). The repo bundles a `music-caption-rewriter` agent skill that expands a brief description into that format offline, with no external APIs, installable via `npx skills add`. Serving runs through **SGLang-Omni** (`sgl-omni serve --model-path MiniMaxAI/MiniMax-Music3`), which exposes the shared speech API — lyrics in `input`, music description in `instructions`. The constraints are stated plainly and they are real. Inference needs **two CUDA GPUs**: one for Qwen3 and the eight-codebook RVQ autoregressive generation, one for Flow Matching and waveform decoding. Only **non-streaming** generation is supported. Prompts cap at **5,000 tokens** and audio at **9,000 acoustic frames**. And MiniMax is candid that section tags and descriptions provide generative control rather than strict symbolic guarantees — requested tempo, key, instrumentation, and structure may not always land exactly. Licensing is not a standard OSI grant either: the weights ship under a custom **MiniMax-Music3 Community License** rather than Apache or MIT, and the GitHub repository carries no LICENSE file of its own.