Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
**JoyAI-Video-Edit** is JD's official release of a real-time, instruction-guided video editing system for open-ended video streams, and it has gathered **1,485 stars and 60 forks** since the repository appeared on **4 August 2026**. The premise is a genuine departure from how open-source video editing models normally work: given a live camera stream or an uploaded clip plus a natural-language edit instruction, it edits frames **causally as they arrive** — without waiting for the full video, without requiring a predefined video length, and without revisiting future frames. The team frames this as pushing video editing away from offline batch processing and toward interactive streaming generation. The architecture is a three-part stack: an **MLLM-based condition encoder**, a **causal video VAE**, and a **16B-parameter multimodal diffusion transformer (MMDiT)** backbone. It is trained and deployed as an autoregressive diffusion editor, then accelerated with a set of techniques the technical report spells out — aligned autoregressive distribution matching distillation, long-horizon optimization, **bounded KV-state inference**, and deployment-oriented scheduling. The last two matter most in practice: bounded KV state keeps per-chunk compute stable so throughput does not degrade as a stream runs long, and the distillation work is what reduces train-inference mismatch and the accumulated temporal drift that usually wrecks long autoregressive video rollouts. On throughput, the repository reports that the full end-to-end pipeline reaches **30 FPS at 720 x 1248** in their deployment benchmark. The separately published live demo figure is more sobering and more useful for anyone budgeting hardware: on a **single RTX PRO 6000 (Blackwell)** GPU, the demo runs **840 x 480 at 24 FPS or 720p at 16 FPS**. Those are two different measurements — the 30 FPS number is the deployment pipeline benchmark, the 16-24 FPS numbers are what one Blackwell card delivers in the public demo — and it is worth keeping them apart when judging what the system will do on your own machine. The editing surface is broad rather than task-specific. Supported instruction types include subject edits, local object edits, subject add/remove/replace, background replacement, style transfer, motion changes, and **reference-image-guided editing (RV2V)**. RV2V is where the project has moved fastest: an upgraded checkpoint landed on **14 August 2026** specifically to strengthen it, with better subject and identity preservation, more faithful reference conditioning, and improved temporal consistency across long streams. The weights are on Hugging Face at `jdopensource/JoyAI-Video-Edit` (DiT checkpoint plus VAE), and a **streaming V2V demo Space** is live. The honest limitations are mostly about access rather than quality. Consumer-GPU support is explicitly still on the TODO list — optimizing for cards like the GeForce RTX 5090 has not happened yet, so an RTX PRO 6000 class GPU is the realistic entry point today. A **Diffusers** pipeline is also listed as future work, meaning setup currently runs through the repo's own conda environment and `deploy/run_server.sh` rather than a familiar one-line loader, with MiMo-VL and ONNX detector files as external runtime dependencies. Most significantly, **only the deployment code is open** — the full training framework and data generation pipeline are promised but not yet released, so the repository is presently something you run rather than something you retrain. Code and weights are **Apache-2.0**, and the technical report is on arXiv (2608.03974).