Edge-preserving Near-light Photometric Stereo with Neural Surfaces

This paper presents a near-light photometric stereo method that faithfully preserves sharp depth edges in the 3D reconstruction. Unlike previous methods that rely on finite differentiation for approximating depth partial derivatives and surface normals, we introduce an analytically differentiable neural surface in near-light photometric stereo for avoiding differentiation errors at sharp depth edges, where the depth is represented as a neural function of the image coordinates. By further formulating the Lambertian albedo as a dependent variable resulting from the surface normal and depth, our method is insusceptible to inaccurate depth initialization. Experiments on both synthetic and real-world scenes demonstrate the effectiveness of our method for detailed shape recovery with edge preservation.

[1]  Paul Debevec,et al.  NeRFactor , 2021, ACM Trans. Graph..

[2]  Ignas Budvytis,et al.  LUCES: A Dataset for Near-Field Point Light Source Photometric Stereo , 2021, BMVC.

[3]  Noah Snavely,et al.  PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Ignas Budvytis,et al.  A CNN Based Approach for the Near-Field Photometric Stereo Problem , 2020, BMVC.

[5]  Gordon Wetzstein,et al.  Implicit Neural Representations with Periodic Activation Functions , 2020, NeurIPS.

[6]  Thomas Funkhouser,et al.  Local Deep Implicit Functions for 3D Shape , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Duygu Ceylan,et al.  DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction , 2019, NeurIPS.

[8]  Zhe Wu,et al.  A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo , 2019, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Richard A. Newcombe,et al.  DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Chao Liu,et al.  Near-light photometric stereo using circularly placed point light sources , 2018, 2018 IEEE International Conference on Computational Photography (ICCP).

[11]  Takanori Maehara,et al.  Neural Inverse Rendering for General Reflectance Photometric Stereo , 2018, ICML.

[12]  Jean-Denis Durou,et al.  Normal Integration: A Survey , 2017, Journal of Mathematical Imaging and Vision.

[13]  Daniel Cremers,et al.  LED-Based Photometric Stereo: Modeling, Calibration and Numerical Solution , 2017, Journal of Mathematical Imaging and Vision.

[14]  Roberto Cipolla,et al.  Semi-Calibrated Near Field Photometric Stereo , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Zhan Song,et al.  A novel photometric stereo method with nonisotropic point light sources , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[16]  Roberto Cipolla,et al.  A Single-Lobe Photometric Stereo Approach for Heterogeneous Material , 2016, SIAM J. Imaging Sci..

[17]  Xiang Huang,et al.  A Streamlined Photometric Stereo Framework for Cultural Heritage , 2016, ECCV Workshops.

[18]  Daniel Cremers,et al.  Realistic photometric stereo using partial differential irradiance equation ratios , 2015, Comput. Graph..

[19]  Gregory H. Bearman,et al.  Near light correction for image relighting and 3D shape recovery , 2015, 2015 Digital Heritage.

[20]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[21]  Alfred M. Bruckstein,et al.  Near Field Photometric Stereo with Point Light Sources , 2014, SIAM J. Imaging Sci..

[22]  Jiuai Sun,et al.  An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region , 2014, Pattern Recognit. Lett..

[23]  Yasuyuki Matsushita,et al.  Calibrating a Non-isotropic Near Point Light Source Using a Plane , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Charlie C. L. Wang,et al.  Surface-from-Gradients: An Approach Based on Discrete Geometry Processing , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Benjamin Bringier,et al.  Tridimensional reconstruction by photometric stereo with near spot light sources , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[26]  Adrien Bartoli,et al.  3D Reconstruction in Laparoscopy with Close-Range Photometric Stereo , 2012, MICCAI.

[27]  Baining Guo,et al.  All-frequency rendering of dynamic, spatially-varying reflectance , 2009, ACM Trans. Graph..

[28]  Peter Kovesi,et al.  Shapelets correlated with surface normals produce surfaces , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[29]  Shree K. Nayar,et al.  Generalization of Lambert's reflectance model , 1994, SIGGRAPH.

[30]  James J. Clark Active photometric stereo , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Yuji Iwahori,et al.  Reconstructing shape from shading images under point light source illumination , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[32]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[34]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[35]  Yasuyuki Matsushita,et al.  Deep Near-Light Photometric Stereo for Spatially Varying Reflectances , 2020, ECCV.

[36]  Brent Burley Physically-Based Shading at Disney , 2012 .