Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Prism is a dynamic sparse attention framework for natively training joint video-audio generation models at 2K, developed by researchers from Fudan University, Tencent Hunyuan and Zhejiang University. The authors argue that full attention has quadratic cost and, as resolution increases, spreads attention over increasingly redundant tokens, and that existing sparse attention methods either target training-free acceleration or overlook how cross-modal interactions concentrate around sound-producing regions. Prism organizes the token sequence into spatiotemporal macro-zones, estimates each zone's local information structure from video feature variance along the channel and feature norms from the audio-to-video cross-attention, and assigns each zone a tailored block shape, with finer partitioning along axes of rapid visual change and strong audio-visual coupling. A hybrid block selection strategy then sets per-query sparsity. The authors report a 2.5x training speedup compared to full attention while surpassing it in generation quality. The repository provides data pre-processing code for latent extraction and decoding, training code, full finetuning code and inference code, plus two preview checkpoints, Prism-preview-alpha (stable) and Prism-preview-beta (motion), built on the MOVA-360p base model. Per the README, the preview model supports native joint video-audio generation and training at 720p, 1080p and 2K. The README lists 720p training at a minimum of 32 NVIDIA 80 GB GPUs and 1080p or 2K training at a minimum of 64; for inference, 720p can run on a single 80 GB GPU while 1080p and 2K need four or more. The planned Prism-pro model is not yet released. Prism's code and weights are under the MIT license, while the MOVA-360p base model is covered by Apache-2.0.
hacksider
Real-time AI face swap and one-click video deepfake with only a single image
harry0703
AI-powered short video generator that automates scripting, footage sourcing, subtitles, and composition — supporting 10+ LLM providers and batch production.