SurfaceNet: Adversarial SVBRDF Estimation from a Single Image

In this paper we present SurfaceNet, an approach for estimating spatially-varying bidirectional reflectance distribution function (SVBRDF) material properties from a single image. We pose the problem as an image translation task and propose a novel patch-based generative adversarial network (GAN) that is able to produce high-quality, high-resolution surface reflectance maps. The employment of the GAN paradigm has a twofold objective: 1) allowing the model to recover finer details than standard translation models; 2) reducing the domain shift between synthetic and real data distributions in an unsupervised way. An extensive evaluation, carried out on a public benchmark of synthetic and real images under different illumination conditions, shows that SurfaceNet largely outperforms existing SVBRDF reconstruction methods, both quantitatively and qualitatively. Furthermore, SurfaceNet exhibits a remarkable ability in generating high-quality maps from real samples without any supervision at training time.

Overview of the SurfaceNet framework. An input image is fed to the generator, which estimates SVBRDF parameter maps. A discriminator receives patches of SVBRDF maps and attempts to distinguish between estimated maps (from both real and synthetic images) and ground-truth maps (for synthetic images only). A supervised loss term (based on L1 norm and MS-SSIM similarity) is computed on the output maps from the generator using ground-truth SVBRDF maps. An adversarial unsupervised loss term is instead computed for the patch discriminator. Circled “C” blocks indicate feature concatenation.