Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Realtime-Venus is a full-duplex interaction system from Ant Group's Venus Team, built with Tsinghua University; its GitHub repository was created September 16, 2026. It combines two 9B streaming checkpoints with an asynchronous execution runtime, Realtime-Venus-Harness. Realtime-Venus-Omni is a conversational model for streaming audio and video that decides when to respond and generates text and native speech on a shared causal timeline; adapted from MiniCPM-o 4.5 / Omni-Flow, it has a SigLIP2 visual encoder, a Whisper-Medium audio encoder, a Qwen3-8B language backbone, discrete S3 speech tokens with a streaming flow-matching decoder, a 40,960-token context length, and BF16 weights. Realtime-Venus-Audio is a separately trained checkpoint on the same backbone (Whisper-Medium encoder, Qwen3-8B, same context length and dtype, no visual encoder at inference) for audio understanding and audio-driven text or speech conversation. Harness captures in-stream <delegate> requests, dispatches them through a Multimodal/direct-answer path, a General path for tool- or multi-step tasks, or a registered Skill path, and returns results via <backend> tags for Omni to speak; it installs standalone with Python 3.11 or 3.12. The source release includes the Omni integration, the Harness package, and a browser demo using a Codex task backend (its microphone-only mode also uses Omni); both checkpoints' weights and custom Transformers code sit in one inclusionAI/Realtime-Venus Hugging Face repository, fetched by a unified download_models.py script or a ModelScope mirror, and Audio also ships standalone offline- and duplex-chat examples. Standalone inference needs Python 3.10, CUDA, and FFmpeg; the browser demo's installer targets Linux, its online Duplex mode needs at least one NVIDIA A100 GPU, and the embedded Codex backend requires network access and a working Codex login. The browser demo takes voice, camera-plus-microphone, or uploaded video (up to 200 MB) and serves one active conversation. A separate Beta Android demo APK (tag android-beta-0918) is offered for research and demonstration. The paper is posted as arXiv 2609.13814. Source code is Apache License 2.0, except third-party fonts and paper figures, which keep their own licenses; the Hugging Face repository lists the same license.