Marigold Normals + Intrinsics v1.1 — Unified Surface Geometry & Lighting Decomposition
Replicate Wrapper · Single-Image Scene Understanding
This model combines two official Marigold pipelines from ETH Zürich:
Marigold Normals v1.1 – Diffusion-based dense surface normal estimation
Marigold IID Lighting v1.1 – Intrinsic image decomposition (Albedo, Shading, Residual)
It exposes a simple image-to-multi-image API on Replicate. Upload one RGB image, and the model outputs both geometry (normals) and intrinsics (albedo, shading, residual).
What this unified model does
Given a single RGB input image, the model performs:
- Surface Normal Estimation
Powered by MarigoldNormalsPipeline, it predicts dense per-pixel surface normals in camera space and returns them as an RGB-encoded normal map:
normals ∈ [-1, 1]³ → mapped to [0, 255] RGB
suitable for 3D reconstruction, relighting, segmentation, material replacement, etc.
- Intrinsic Image Decomposition
Using Marigold IID Lighting, it produces:
albedo.png — lighting-independent surface color
shading.png — illumination and geometric shading
residual.png — remaining signal the IID model cannot fully explain
These outputs are physically meaningful and compatible with workflows such as relighting, room visualization, material replacement, and AR/VR scene understanding.
Typical Use Cases 🏠 Room / Interior Understanding
Derive wall/floor/ceiling normals for 3D alignment
Replace materials realistically using albedo + normals
Estimate lighting direction from shading maps
🏗️ 3D Reconstruction / Calibration
Use normals to detect plane orientation & geometry consistency
Feed normals + albedo into inverse rendering pipelines
🖌️ Relighting & Material Editing
Modify albedo while keeping shading intact
Relight scenes using shading maps
Use normals for environment-light estimation
👓 AR/VR Perception
Extract geometry + lighting cues from a single image
Build lightweight perception for robotics or XR applications
Filename Description normals.png RGB visualization of estimated surface normals. albedo.png Reflectance-only surface color (lighting removed). shading.png Illumination component only. residual.png Remaining unexplained signal.
Implementation Details Upstream Models
The wrapper loads both official models:
Component Upstream Source Normals prs-eth/marigold-normals-v1-1 Intrinsics prs-eth/marigold-iid-lighting-v1-1
Both are part of the ETH Zürich Marigold project.
Device & Precision
Uses CUDA + float16 if available
Falls back to CPU + float32
Ensures all outputs are clean RGB PNGs
Robust Output Extraction
The wrapper gracefully handles differences in Diffusers API:
tries .prediction,
then .pred_normals,
then .images[0].
Intrinsic Layers
The model returns:
albedo = surface reflectance
shading = lighting × geometry
residual = leftover information
These three layers recombine approximately to the original image.
Limitations
Works best on natural interior/exterior scenes with real lighting.
Noisy, low-light, or stylized images may degrade prediction quality.
Normal estimation may be unstable on texture-less regions (white walls, glass).
Intrinsics are not physically perfect but strongly useful for editing workflows.
License
Both upstream models are released under:
CreativeML Open RAIL++-M License (See the exact terms linked on the Hugging Face model pages.)
By using this unified Replicate model, you agree to comply with the upstream license, including restrictions on:
redistribution
commercial use
content limitations
Please review the license for your use case.
Liscence https://huggingface.co/prs-eth/marigold-normals-v1-1/blob/main/README.md https://huggingface.co/prs-eth/marigold-iid-lighting-v1-1/blob/main/README.md