Learning to estimate surface normal via deep photometric stereo networks

Abstract Photometric stereo, aiming at estimating the surface normal of an object from a set of images under different illumination conditions, has gained a lot of attention recently. However, most existing state-of-the-art works of photometric stereo heavily rely on elaborately light calibration which limit the practical application of this technology. In this paper, we propose a self-calibrating photometric stereo method which could accurately reconstruction the surface normal. Specifically, a two-stage deep architecture is developed to perform light calibration and surface normal estimation simultaneously. Extensively experiment results on public available real datasets demonstrate that our model could estimate surface normal more accurately than most state-of-the-arts.

[1]  Kai Han,et al.  PS-FCN: A Flexible Learning Framework for Photometric Stereo , 2018, ECCV.

[2]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[3]  Paolo Favaro,et al.  A Closed-Form, Consistent and Robust Solution to Uncalibrated Photometric Stereo Via Local Diffuse Reflectance Maxima , 2013, International Journal of Computer Vision.

[4]  Steven M. Seitz,et al.  Example-based photometric stereo: shape reconstruction with general, varying BRDFs , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Roberto Cipolla,et al.  Multiview Photometric Stereo , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Ping Tan,et al.  A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo , 2016, CVPR.

[7]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[8]  P. Bicheron,et al.  Bidirectional reflectance distribution function signatures of major biomes observed from space , 2000 .

[9]  Athinodoros S. Georghiades,et al.  Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Yasuyuki Matsushita,et al.  Uncalibrated Photometric Stereo Under Natural Illumination , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[11]  Yannick Hold-Geoffroy,et al.  Deep Outdoor Illumination Estimation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Yasuyuki Matsushita,et al.  Photometric Stereo Under Non-uniform Light Intensities and Exposures , 2016, ECCV.

[13]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Martin Weiß,et al.  An improvement of the convergence proof of the ADAM-Optimizer , 2018, ArXiv.

[15]  Jean-François Lalonde,et al.  Learning to Estimate Indoor Lighting from 3D Objects , 2018, 2018 International Conference on 3D Vision (3DV).

[16]  Michael Goesele,et al.  A Survey of Photometric Stereo Techniques , 2015, Found. Trends Comput. Graph. Vis..

[17]  Mike J. Chantler,et al.  Can two specular pixels calibrate photometric stereo? , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[18]  Steven M. Seitz,et al.  Shape and spatially-varying BRDFs from photometric stereo , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[19]  David J. Kriegman,et al.  Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Yasuyuki Matsushita,et al.  Self-calibrating photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Yasuyuki Matsushita,et al.  Learning to Minify Photometric Stereo , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Hideki Hayakawa.  Photometric stereo under a light source with arbitrary motion , 1994 .

[23]  Ersin Yumer,et al.  Learning to predict indoor illumination from a single image , 2017, ACM Trans. Graph..

[24]  Yoichi Sato,et al.  SymPS: BRDF Symmetry Guided Photometric Stereo for Shape and Light Source Estimation , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[26]  Yasuyuki Matsushita,et al.  Self-Calibrating Deep Photometric Stereo Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).