On September 15, 2026, NVIDIA released the open weights of two 3D perception models from its Vision AI lineup on Hugging Face: FoundationStereo for depth estimation from stereo frames and FoundationPose for determining 6-DoF object pose. Both are distributed under the NVIDIA Open Model License with permission for commercial use, operate zero-shot without fine-tuning, and are designed for hardware ranging from Jetson to A100 and H100.

image
image
image

What happened

On September 15, 2026, NVIDIA released two open 3D perception models from its Vision AI lineup on Hugging Face. The first is FoundationStereo, repository nvidia/c-foundationstereo-s, 63 million parameters: from a pair of RGB frames from a stereo camera, it outputs a disparity map in zero-shot, without fine-tuning. The second is FoundationPose, repository nvidia/foundationpose: from a textured CAD model of an object in OBJ format and an RGBD frame, it estimates the object's position and orientation in six degrees of freedom. Both models are marked as ready for commercial use and are provided in ONNX and TensorRT formats. FoundationStereo has been tested on hardware from Jetson AGX Orin and Thor to A100 and H100; on A100 in FP16 mode at a resolution of 320x736, inference takes 199 ms per frame.

Context

Both models are not new developments of 2026: their scientific basis has been peer-reviewed at specialized conferences. The FoundationStereo architecture was published at CVPR 2025 as an Oral presentation with a Best Paper nomination, and FoundationPose was highlighted at CVPR 2024. Since the weights are open, the metrics claimed in the papers are reproducible by the community — a rare case where the conclusions of a paper can be directly verified on one's own hardware. FoundationStereo combines a transformer and CNN and relies on the monocular prior DepthAnythingV2; it was trained on 1.6 million synthetic stereo pairs FSD generated in Omniverse, plus the CREStereo and Tartanair datasets. FoundationPose was trained only on synthetic data: Objaverse with 40,000 objects and Google Scanned Objects, rendered in Omniverse and Isaac Sim. On BOP benchmarks, it shows an average score of 83.3, including 83.0 on TLESS and 88.0 on YCB-Video. The bet on synthetic data instead of collecting real datasets is a key element of the approach: both models should transfer to new scenes and objects without fine-tuning.

Why this matters for the industry

For robotics teams and startups, the main barrier to product perception disappears — collecting datasets and fine-tuning stereo depth and pose capture pipelines: zero-shot inference on new scenes and objects works on edge devices from Jetson Orin Nano to H100, and the license explicitly permits commercial application. The path from prototype to industrial pilot of recognition and manipulation becomes noticeably shorter and cheaper. On the other hand, the protective barrier of teams whose product was fine-tuned depth and pose models blurs — closed vendors distributing such models under paid licenses will face price pressure. The composition of product input data also changes: value shifts from model training to the quality of CAD assets, sensor calibration, and orchestration around perception. If the "synthetic instead of real datasets" pattern takes hold, expect waves of integrations into robotics stacks, fine-tune versions on top of open weights, and independent measurements on real production scenes.

Why this matters for users

The weights can be downloaded for free from Hugging Face and tested on one's own hardware the same day. For FoundationStereo, a stereo camera and its calibration are enough — the model will build a depth map from a pair of RGB frames; for FoundationPose, a textured CAD model in OBJ and an RGBD frame are needed instead of a collected training set. A realistic plan for the first one to two weeks is to deploy TensorRT inference and build a small internal eval set on one's own scenes; in one sprint, it is realistic to build a demo of volume measurement or 6-DoF object tracking. Depth maps and pose tracking are suitable for robotics and smart space projects, so it makes sense to test on scenes and materials typical for you.

What is still unknown / limitations

Zero-shot in 3D perception is not equivalent to the shift opened by LLMs: the result critically depends on the quality of the sensor, stereo pair calibration, and RGBD depth accuracy, not just the weights themselves. Problem areas for both models are transparent and shiny objects, overexposure, and poor lighting. Published metrics — average score on BOP benchmarks and latency on A100 — were obtained on benchmarks and do not guarantee the same results on specific hardware and specific scenes; independent comparative measurements on real production scenes are currently lacking. Expectations of a wave of integrations, niche products, and a change in the default robotics stack are an interpretation of the trend, not a confirmed fact.

Sources

Author

Look at AI, editorial team