Deep Near-Light Photometric Stereo for Spatially Varying Reflectances

This paper presents a near-light photometric stereo method for spatially varying reflectances. Recent studies in photometric stereo proposed learning-based approaches to handle diverse real-world reflectances and achieve high accuracy compared to conventional methods. However, they assume distant (i.e., parallel) lights, which can in practical settings only be approximately realized, and they fail in near-light conditions. Near-light photometric stereo methods address near-light conditions but previous works are limited to over-simplified reflectances, such as Lambertian reflectance. The proposed method takes a hybrid approach of distantand near-light models, where the surface normal of a small area (corresponding to a pixel) is computed locally with a distant light assumption, and the reconstruction error is assessed based on a near-light image formation model. This paper is the first work to solve unknown, spatially varying, diverse reflectances in near-light photometric stereo.

[1]  Zhe Wu,et al.  A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo , 2019, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Takanori Maehara,et al.  Neural Inverse Rendering for General Reflectance Photometric Stereo , 2018, ICML.

[3]  Katsushi Ikeuchi,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Bi-polynomial Modeling of Low-frequency Reflectances , 2022 .

[4]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[5]  Alfred M. Bruckstein,et al.  Near Field Photometric Stereo with Point Light Sources , 2014, SIAM J. Imaging Sci..

[6]  Adrien Bartoli,et al.  3D Reconstruction in Laparoscopy with Close-Range Photometric Stereo , 2012, MICCAI.

[7]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[8]  Brent Burley.  Physically-Based Shading at Disney , 2012 .

[9]  Andrea Torsello,et al.  A Scale Independent Selection Process for 3D Object Recognition in Cluttered Scenes , 2013, International Journal of Computer Vision.

[10]  Jean-Denis Durou,et al.  Variational Methods for Normal Integration , 2017, Journal of Mathematical Imaging and Vision.

[11]  Imari Sato,et al.  A Microfacet-Based Reflectance Model for Photometric Stereo with Highly Specular Surfaces , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Daniel Cremers,et al.  LED-Based Photometric Stereo: Modeling, Calibration and Numerical Solution , 2017, Journal of Mathematical Imaging and Vision.

[13]  Daniel Cremers,et al.  Realistic photometric stereo using partial differential irradiance equation ratios , 2015, Comput. Graph..

[14]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[15]  Gregory H. Bearman,et al.  Near light correction for image relighting and 3D shape recovery , 2015, 2015 Digital Heritage.

[16]  Yasuyuki Matsushita,et al.  Light Structure from Pin Motion: Geometric Point Light Source Calibration , 2020, International Journal of Computer Vision.

[17]  Xin Pei,et al.  Calibration of position and orientation for point light source synchronously with single image in photometric stereo. , 2019, Optics express.

[18]  Jiuai Sun,et al.  An improved photometric stereo through distance estimation and light vector optimization from diffused maxima region , 2014, Pattern Recognit. Lett..

[19]  Athinodoros S. Georghiades,et al.  Incorporating the Torrance and Sparrow model of reflectance in uncalibrated photometric stereo , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Yuji Iwahori,et al.  Reconstructing shape from shading images under point light source illumination , 1990, [1990] Proceedings. 10th International Conference on Pattern Recognition.

[21]  Daniel Cremers,et al.  A Non-convex Variational Approach to Photometric Stereo under Inaccurate Lighting , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[23]  Benjamin Bringier,et al.  Tridimensional reconstruction by photometric stereo with near spot light sources , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[24]  MatusikWojciech,et al.  A data-driven reflectance model , 2003 .

[25]  Edward H. Adelson,et al.  Shape estimation in natural illumination , 2011, CVPR 2011.

[26]  Zhan Song,et al.  A novel photometric stereo method with nonisotropic point light sources , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[27]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[28]  Simon Fuhrmann,et al.  Geometric Point Light Source Calibration , 2013, VMV.

[29]  Yvain Quéau,et al.  Unifying diffuse and specular reflections for the photometric stereo problem , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[30]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[31]  Satoshi Ikehata,et al.  CNN-PS: CNN-based Photometric Stereo for General Non-Convex Surfaces , 2018, ECCV.

[32]  Yasuyuki Matsushita,et al.  Deep Photometric Stereo Network , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[33]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[34]  Kai Han,et al.  PS-FCN: A Flexible Learning Framework for Photometric Stereo , 2018, ECCV.

[35]  Yasuyuki Matsushita,et al.  Calibrating a Non-isotropic Near Point Light Source Using a Plane , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.