Unbiased Photometric Stereo for Colored Surfaces: A Variational Approach

3D shape recovery using photometric stereo (PS) gained increasing attention in the computer vision community in the last three decades due to its ability to recover the thinnest geometric structures. Yet, the reliability of PS for color images is difficult to guarantee, because existing methods are usually formulated as the sequential estimation of the colored albedos, the normals and the depth. Hence, the overall reliability depends on that of each subtask. In this work we propose a new formulation of color photometric stereo, based on image ratios, that makes the technique independent from the albedos. This allows the unbiased 3D-reconstruction of colored surfaces in a single step, by solving a system of linear PDEs using a variational approach.

[1]  D. Hinkley.  On the ratio of two correlated normal random variables , 1969 .

[2]  Björn Stenger,et al.  Color photometric stereo for multicolored surfaces , 2011, 2011 International Conference on Computer Vision.

[3]  Michael Breuß,et al.  Perspective Shape from Shading: Ambiguity Analysis and Numerical Approximations , 2012, SIAM J. Imaging Sci..

[4]  Jean-Denis Durou,et al.  Solving Uncalibrated Photometric Stereo Using Total Variation , 2014, Journal of Mathematical Imaging and Vision.

[5]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[6]  Tai-Pang Wu,et al.  Dense photometric stereo using a mirror sphere and graph cut , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  David J. Kriegman,et al.  Toward Reconstructing Surfaces With Arbitrary Isotropic Reflectance : A Stratified Photometric Stereo Approach , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[8]  Ravi Ramamoorthi,et al.  On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Jean-Denis Durou,et al.  A L^1 -TV Algorithm for Robust Perspective Photometric Stereo with Spatially-Varying Lightings , 2015, SSVM.

[10]  Maurizio Falcone,et al.  Uniqueness and Approximation of a Photometric Shape-from-Shading Model , 2013, SIAM J. Imaging Sci..

[11]  Alfred M. Bruckstein,et al.  Integrability disambiguates surface recovery in two-image photometric stereo , 1992, International Journal of Computer Vision.

[12]  Jian Wang,et al.  Photometric Stereo with Small Angular Variations , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[13]  David J. Kriegman,et al.  Photometric Stereo in a Scattering Medium , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Moshe Ben-Ezra,et al.  Photometric Stereo for Dynamic Surface Orientations , 2010, ECCV.

[15]  Yoav Y. Schechner,et al.  Multiplexed fluorescence unmixing , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[16]  Dmitry B. Goldgof,et al.  A Simple Strategy for Calibrating the Geometry of Light Sources , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Edward H. Adelson,et al.  Microgeometry capture using an elastomeric sensor , 2011, ACM Trans. Graph..

[18]  Matthew Harker,et al.  Least squares surface reconstruction from gradients: Direct algebraic methods with spectral, Tikhonov, and constrained regularization , 2011, CVPR 2011.

[19]  L. L..  Reconstruction of shape from shading in color images , .

[20]  Katsushi Ikeuchi,et al.  Consensus photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[21]  Pierre Gurdjos,et al.  A full photometric and geometric model for attached webcam/matte screen devices , 2016, Signal Process. Image Commun..

[22]  Alfred M. Bruckstein,et al.  A Direct Differential Approach to Photometric Stereo with Perspective Viewing , 2014, SIAM J. Imaging Sci..

[23]  Olivier D. Faugeras,et al.  "Perspective shape from shading" and viscosity solutions , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Alfred M. Bruckstein,et al.  RGBD-fusion: Real-time high precision depth recovery , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Jan Chabrowski,et al.  On variational approach to photometric stereo , 1993 .

[26]  In-So Kweon,et al.  One-day outdoor photometric stereo via skylight estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Nahum Kiryati,et al.  Photometric stereo under perspective projection , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[29]  Alfred M. Bruckstein,et al.  Near Field Photometric Stereo with Point Light Sources , 2014, SIAM J. Imaging Sci..

[30]  H. P..  Annales de l'Institut Henri Poincaré , 1931, Nature.

[31]  Adrien Bartoli,et al.  3D Reconstruction in Laparoscopy with Close-Range Photometric Stereo , 2012, MICCAI.

[32]  Olivier D. Faugeras,et al.  Shape From Shading , 2006, Handbook of Mathematical Models in Computer Vision.

[33]  William A. P. Smith,et al.  Height from photometric ratio with model-based light source selection , 2016, Comput. Vis. Image Underst..

[34]  Tony F. Chan,et al.  Structure-Texture Image Decomposition—Modeling, Algorithms, and Parameter Selection , 2006, International Journal of Computer Vision.

[35]  Paolo Favaro,et al.  Uncalibrated Near-Light Photometric Stereo , 2014, BMVC.

[36]  Ping Tan,et al.  Ring-Light Photometric Stereo , 2010, ECCV.

[37]  Michael Brady,et al.  Integrating stereo and photometric stereo to monitor the development of glaucoma , 1990, BMVC.

[38]  Björn Stenger,et al.  Augmenting Depth Camera Output Using Photometric Stereo , 2011, MVA.

[39]  David J. Kriegman,et al.  Beyond Lambert: reconstructing specular surfaces using color , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[40]  Daniel Snow,et al.  Shape and albedo from multiple images using integrability , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Katsushi Ikeuchi,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Bi-polynomial Modeling of Low-frequency Reflectances , 2022 .

[42]  Charlie C. L. Wang,et al.  Photometric stereo with near point lighting: A solution by mesh deformation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Kiyoharu Aizawa,et al.  Robust photometric stereo using sparse regression , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Rama Chellappa,et al.  Direct Analytical Methods for Solving Poisson Equations in Computer Vision Problems , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Ye Duan,et al.  Color Photometric Stereo for Albedo and Shape Reconstruction , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[46]  Björn Stenger,et al.  Video Normals from Colored Lights , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Branislav Jaramaz,et al.  A Multi-Image Shape-from-Shading Framework for Near-Lighting Perspective Endoscopes , 2009, International Journal of Computer Vision.

[48]  Pieter Peers,et al.  Dynamic shape capture using multi-view photometric stereo , 2009, ACM Trans. Graph..

[49]  Paolo Favaro,et al.  A New Perspective on Uncalibrated Photometric Stereo , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Min H. Kim,et al.  Multispectral Photometric Stereo for Acquiring High-Fidelity Surface Normals , 2014, IEEE Computer Graphics and Applications.

[51]  Gary L. Miller,et al.  A Nearly-m log n Time Solver for SDD Linear Systems , 2011, 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science.

[52]  Mark A Neifeld,et al.  Feature-specific imaging. , 2003, Applied optics.

[53]  Yvain Quéau,et al.  Unifying diffuse and specular reflections for the photometric stereo problem , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[54]  Roberto Cipolla,et al.  Overcoming Shadows in 3-Source Photometric Stereo , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Lyndon Smith,et al.  Reflectance of human skin using colour photometric stereo: with particular application to pigmented lesion analysis , 2008, Skin research and technology : official journal of International Society for Bioengineering and the Skin (ISBS) [and] International Society for Digital Imaging of Skin (ISDIS) [and] International Society for Skin Imaging.

[56]  Jean-Denis Durou,et al.  Integrating the Normal Field of a Surface in the Presence of Discontinuities , 2009, EMMCVPR.

[57]  Jitendra Malik,et al.  Shape, albedo, and illumination from a single image of an unknown object , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[58]  Imari Sato,et al.  Color Photometric Stereo Using a Rainbow Light for Non-Lambertian Multicolored Surfaces , 2014, ACCV.

[59]  Maria Petrou,et al.  The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..