Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
TLive-Omni is an omni-modal understanding model from the TaoLiveAIGC team at Taobao, Alibaba, aimed at a domain where the facts about a product are scattered across a host's speech, the video frames, separate product images, and text overlaid on screen. It takes image, video, audio, and text in and returns text only, grafting the pretrained AuT audio encoder from Qwen3-Omni onto a Qwen3.5 vision-language backbone through a lightweight audio aligner, and it accepts up to 256K tokens of multimodal context so a long stream segment fits in a single pass. The architectural contribution the technical report leads with is Per-vGrid, a timestamped token layout that groups each video grid with the audio covering the same time interval inside explicit boundary tokens, keeping narration adjacent to the frames it describes rather than interleaving the two arbitrarily. Training runs in three supervised stages — 5M ASR samples to fit the audio aligner alone, then 26M audio samples that unfreeze the encoder for captioning and audio QA, then 14M mixed audio, image, video, and text samples for the live-commerce capabilities themselves — followed by Faithful-RFT, a reinforcement fine-tuning stage that deliberately suppresses explicit think traces and scores only the final answer against task-verifiable rewards, on the reasoning that a live stream cannot wait for a chain of thought. Two checkpoints, TLive-Omni-4B and TLive-Omni-9B, ship on Hugging Face under Apache-2.0 alongside inference examples, a customized vLLM fork, and a prebuilt wheel tested on NVIDIA H20 GPUs. On the project's own live-commerce evaluation the 9B model reports a 6.46 character error rate on live ASR, below every open-source omni model in its comparison table, where Qwen3-Omni's 30B-A3B sits at 6.75; the 4B variant reports 91.45 product visual grounding AP against 68.88 for Qwen3-Omni, and leads its larger sibling on the TimeLens-Bench temporal grounding split. General benchmarks put the 9B at 88.9 on MMBench and 90.3 on OCRBench. The authors are explicit that the model does understanding only — no generation, no full-duplex real-time interaction — and list robustness on longer, noisier streams as open work.
hacksider
Real-time AI face swap and one-click video deepfake with only a single image
harry0703
AI-powered short video generator that automates scripting, footage sourcing, subtitles, and composition — supporting 10+ LLM providers and batch production.