Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
MOSS-VL is an open-weight vision-language model series from the OpenMOSS team aimed at long-form and real-time video understanding. All three checkpoints are 11B parameters with a 256K context: MOSS-VL-Realtime for continuous video streams, MOSS-VL-Instruct-0708 for offline long-video work and in-depth dialogue, and MOSS-VL-Base-0708 as a pre-trained foundation for continued pre-training and fine-tuning. The architecture departs from the usual 'watch first, answer after' pattern by using cross-attention to decouple visual encoding from language reasoning, which lets perception and text generation run in parallel over an arriving stream. Every sampled frame is anchored with dedicated absolute-timestamp tokens so the model can reason about pacing and duration and adapt to variable frame rates, and XRoPE maps text tokens and video patches into a shared 3D (t, h, w) coordinate space for patch-level and moment-level grounding. The behaviours this buys are the interesting part: a viewer can interrupt at any timestamp and get an answer from the frames received so far, the model can choose to stay silent and keep watching when no key event has occurred, and it revises earlier statements as the feed changes. The project reports 66.0 on PA@OmniMMI for MOSS-VL-Realtime and describes the series as leading open-source results on several streaming video benchmarks; a technical report was published on arXiv in August 2026 (2608.15045). Deployment work has been steady rather than promised: FP8 and NF4 quantized checkpoints for both Instruct-0708 and Realtime landed in August and fit a single 24 GB NVIDIA GPU, followed by a published calibration-free PTQ recipe covering selective layer coverage and KV-cache quantization for Transformers and SGLang. The repository bundles a FlashAttention-3 backend that adds a cross_kv_boundary interface representing each query row's visible KV prefix with a single int32 rather than a dense mask, ships a lightweight SFT framework on the Hugging Face Trainer with full-parameter and LoRA paths, and is supported by SGLang, LlamaFactory, and ms-swift. It is Apache-2.0. Full training code, RL post-training, and task-level cookbooks remain open roadmap items.