Open Source
Explore the latest AI open-source projects from GitHub and HuggingFace.
Explore the latest AI open-source projects from GitHub and HuggingFace.
Marigold V2 ("Revisiting Diffusion Transformers for Monocular Depth Estimation," from Huawei Bayer Lab with EPFL and the University of Bologna) repurposes a frozen Qwen-Image-Edit-2509 diffusion transformer, quantized to 4 bit on load, into a single-step dense predictor via LoRA fine-tuning. The authors say fine-tuning takes less than a week on a single consumer GPU; every released model was trained on one 32 GB GPU at batch size 1. The repository's initial release on 2026-09-08 shipped inference, evaluation, and training code, and the paper is to appear in ACM Transactions on Graphics and at SIGGRAPH Asia 2026. Nine checkpoints, stored as trainables.safetensors (LoRA adapters plus a fine-tuned VAE decoder where trained), are hosted on the huawei-bayerlab/marigold-v2-0 Hugging Face repo: seven depth checkpoints (the default two-stage log-depth model, its stage-1 initialization, linear and disparity ablations, and three see-through variants trained on LayeredDepth-Syn to predict geometry behind glass), plus one surface-normals and one albedo checkpoint. Training configs are included for all of them except the Uniform-layered depth variant. scripts/infer.py selects a checkpoint by modality or Hugging Face path and writes float32 .npy predictions plus PNG visualizations; the README lists Linux, Python 3.10, and a CUDA GPU, with about 17 GB of GPU memory at 1024² and 29 GB at 2048². Self-reported results (no competitor scores in the README) include zero-shot depth AbsRel/δ1 of 3.6/98.0 on NYUv2 under the Pixel-Perfect Depth protocol, surface-normal mean angular error/within-11.25° of 16.6/61.2 on NYUv2, and Hypersim albedo PSNR 20.78, SSIM 0.811, LPIPS 0.195. The maintainers' checklist marks four items as not yet shipped: depth-completion code, see-through evaluation, a Diffusers integration, and a ComfyUI plugin. The README's intro names metric depth completion as an application the models unlock, but that code isn't released yet. Code and weights are Apache-2.0; the Qwen-Image-Edit-2509 backbone and the datasets keep their own licenses.