HUAWEI Bayer Lab, together with EPFL and the University of Bologna, released Marigold V2 — an open system that turns a pretrained diffusion transformer into a one-step monocular depth estimator from a single image, with code and weights published under the Apache-2.0 license.

image
image
image

What happened

The second version of the Marigold system is built on the pretrained Qwen-Image-Edit-2509 model: the backbone is frozen, training uses LoRA adapters of rank 128 on top of a DiT quantized to 4 bits, and distillation from a multi-step flow-matching model enables one-step inference. The paper (arXiv:2609.08084) reports a 16–26% improvement in AbsRel on KITTI and ETH3D over the previous best result, as well as SOTA on normal estimation (NYUv2, ScanNet, iBims-1, Sintel) and image decomposition: albedo with PSNR 20.78, SSIM 0.811, and LPIPS 0.195 on Hypersim. Code, weights, and a web demo are published in the huawei-bayerlab/marigold-v2 repository.

Context

The first version of Marigold showed that a pretrained generative diffusion model can be used for dense predictions, such as depth, instead of specialized regression depth models. Such diffusion approaches had a systemic overhead: inference took dozens of iterations of the diffusion process, and it is precisely at this point that Marigold V2 performs a single forward pass. Evaluation is conducted on standard comparable benchmarks: KITTI and ETH3D for depth, NYUv2, ScanNet, iBims-1, and Sintel for normals, and Hypersim for albedo, so the reported results can be cross-checked against published numbers from other works. The methodological contribution of this version is two specific techniques: aligning the model's internal representations with semantic features (iREPA on DINOv3) and a two-stage fine-tuning with a new Sinkhorn loss (SinkLoss), which presumably produce sharp edges and remove characteristic blurring.

Why this matters for the industry

For the industry, the main signal is the economics of development: a pretrained DiT for image generation has been brought to a SOTA estimator of dense quantities (depth, normals, albedo) via LoRA fine-tuning on a single consumer GPU with 32 GB, with stage 1 taking 160k steps in about 5 days at batch=1. Inference requires about 17 GB of VRAM at 1024x1024 resolution and about 29 GB at 2048x2048, and takes one step instead of dozens of iterations. This reduces the cost of developing and deploying dense prediction for robotics, computational photography, and 3D reconstruction: teams get a ready-made open baseline on which they can immediately run their own depth and normals pipelines, instead of developing a model from scratch.

Why this matters for users

For readers, the model is immediately available: it can be tried in a web demo on Hugging Face or run locally from the open repository, where code and weights are under Apache-2.0. Among the checkpoints is a Log-layered variant that predicts depth “through” transparent objects, for example behind glass, and a new modality for the model can be connected via a YAML config. For creative tasks, this provides ready-made sharp depth maps without the characteristic blurring of fur, foliage, and thin edges seen in older models.

What is still unknown / limitations

All SOTA numbers (16–26% improvement in AbsRel on KITTI and ETH3D, results on normals and albedo) are author-reported results from their own arXiv paper; independent reproduction is not yet available. No published data on inference latency and throughput, and no managed API or SLA for the project; as of publication, this is a research open-source baseline, not a service.

Sources

Author

Look at AI, editorial team