Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Lance is a 3B active-parameter unified multimodal model from ByteDance that handles image and video understanding, generation, and editing inside a single framework. The interesting part is not the capability list — several labs have shipped unified models by now — but the scale it was achieved at. Lance was trained from scratch on a budget of up to 128 A100 GPUs, and it posts benchmark numbers that sit level with, and in places above, unified models carrying twice its parameter count. Released on GitHub and Hugging Face in May 2026 under Apache-2.0, with a technical report on arXiv, it currently sits at roughly 1,290 stars. ## One Model, Seven Tasks Most multimodal stacks in production are assemblies: a VLM for understanding, a diffusion model for generation, a separate editing model bolted alongside. Lance collapses that into one checkpoint with a single inference entry point. The supported task list is `t2i`, `t2v`, `i2v`, `image_edit`, `video_edit`, `x2t_image`, and `x2t_video` — text-to-image, text-to-video, image-to-video, image editing, video editing, and image and video understanding respectively. All of them run through the same `inference_lance.sh` script with a task flag, and a Gradio interface added in late May exposes the same surface in a browser. The project ships two checkpoints, `Lance_3B` for image tasks and `Lance_3B_Video` for video, and the paper frames the training approach as multi-task synergy — a staged recipe where the tasks reinforce each other rather than competing for capacity. That framing is what makes the benchmark results worth reading closely: the claim is not that a 3B model beats a 7B model at any one thing, but that jointly training the tasks recovers enough capability to stay competitive across all of them. ## The Benchmark Picture On GenEval, Lance scores 0.90 overall, matching the 7B TUNA and edging past Mogao at 0.89 and BAGEL at 0.88 — with a 1.00 on single-object, 0.97 on colors, and 0.87 on position. On VBench for video generation it reports a total score of 85.11, above Wan2.1-T2V at 14B (83.69) and Hunyuan Video (83.43), and above every unified model in the comparison table. GEdit-Bench for image editing gives it a 7.30 average, the strongest among the unified models listed and well clear of BAGEL's 6.52, though still behind the 20B Qwen-Image-Edit at 8.01 and GPT Image 1 at 7.49. DPG-Bench is where the picture gets more honest. Lance lands at 84.67 overall — respectable, but behind TUNA (86.76), TUNA-2 (86.54), and InternVL-U (85.18). The subscores show why: 91.07 on Entity and 93.38 on Relation are strong, but Global at 83.89 and Other at 80.80 drag the average down. On GEdit the text-modification subscore of 4.46 is a clear outlier against its otherwise 7-plus columns. These are the seams you would expect from a compact model trained on a constrained compute budget, and the project does not hide them. ## Deployment and Ecosystem Inference needs a GPU with at least 40GB of VRAM, CUDA 12.4 or newer, and a matched flash-attn build — tested combinations are PyTorch 2.8.0 with cu126 and flash-attn 2.8.3, or PyTorch 2.5.1 with cu124 and flash-attn 2.6.3. Video generation caps at 121 frames at 480p. An optional prompt-rewrite path improves text-to-video and image-to-video quality by routing prompts through an external OpenAI-compatible endpoint, configured through environment variables. The ecosystem work has moved faster than the model itself. Lance gained vLLM-Omni support in early June, which matters more for practical deployment than any benchmark row, and fine-tuning code landed on June 17 with a documented training guide. A Hugging Face Space maintained with help from the HF team gives a zero-install way to evaluate output quality before committing to a 40GB GPU. ## Pros and Cons The strengths are parameter efficiency and breadth. Seven tasks from one 3B checkpoint under Apache-2.0 is a genuinely useful package for teams that cannot host three separate models, and VBench and GEdit results that beat larger unified competitors suggest the multi-task recipe is doing real work. Released weights, released training code, an arXiv report, and vLLM-Omni integration make it reproducible rather than a demo. The limits are stated plainly by the maintainers, which is worth crediting. This is a research artifact, not a product model — output quality varies by prompt, resolution, duration, and motion complexity. The 768x768 image and 480p 12 FPS video ceilings are well below what commercial generators deliver. The 40GB VRAM floor rules out consumer hardware. DPG-Bench and the GEdit text-modification subscore show concrete capability gaps. And the project has six contributors and no tagged releases, with the last push on July 14 — small-team cadence with the maintenance risk that implies. ## Outlook The pattern Lance is testing — whether joint multi-task training lets a small unified model punch above its parameter count — is one of the more consequential open questions in multimodal research right now, because the alternative is that every capability costs its own model. Its VBench and GEdit results are a real data point in favor. Whether that generalizes past 3B, or past a 128-GPU training budget, is exactly what the released fine-tuning code invites the community to find out. ## Conclusion Lance is worth a look for researchers studying unified multimodal architectures and for teams that need several modalities from one deployable checkpoint on a single 40GB GPU. Anyone who needs production-grade fidelity, high-resolution output, or long video should treat it as a reference implementation rather than a drop-in — the maintainers say as much themselves, and the resolution ceilings make the point concrete.
hacksider
Real-time AI face swap and one-click video deepfake with only a single image
harry0703
AI-powered short video generator that automates scripting, footage sourcing, subtitles, and composition — supporting 10+ LLM providers and batch production.