Height from photometric ratio with model-based light source selection

New photometric stereo method that recovers surface height directly.Propose a model-based approach to selecting and removing noisy observations.Exploit photometric ratios to express height recovery as a large sparse linear system of equations.Evaluate quantitatively and qualitatively on wide range of objects including shadows and specularities. In this paper, we present a photometric stereo algorithm for estimating surface height. We follow recent work that uses photometric ratios to obtain a linear formulation relating surface gradients and image intensity. Using smoothed finite difference approximations for the surface gradient, we are able to express surface height recovery as a linear least squares problem that is large but sparse. In order to make the method practically useful, we combine it with a model-based approach that excludes observations which deviate from the assumptions made by the image formation model. Despite its simplicity, we show that our algorithm provides surface height estimates of a high quality even for objects with highly non-Lambertian appearance. We evaluate the method on both synthetic images with ground truth and challenging real images that contain strong specular reflections and cast shadows.

[1]  Stefanos Zafeiriou,et al.  A sparse representation method for determining the optimal illumination directions in Photometric Stereo , 2013, Signal Process..

[2]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[3]  Paolo Favaro,et al.  Uncalibrated Near-Light Photometric Stereo , 2014, BMVC.

[4]  Maurizio Falcone,et al.  Uniqueness and Approximation of a Photometric Shape-from-Shading Model , 2013, SIAM J. Imaging Sci..

[5]  Timothy A. Davis,et al.  Algorithm 915, SuiteSparseQR: Multifrontal multithreaded rank-revealing sparse QR factorization , 2011, TOMS.

[6]  Mubarak Shah,et al.  Shape from shading using linear approximation , 1994, Image Vis. Comput..

[7]  Sang Wook Lee,et al.  Photometric Stereo from Maximum Feasible Lambertian Reflections , 2010, ECCV.

[8]  Rama Chellappa,et al.  A Method for Enforcing Integrability in Shape from Shading Algorithms , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Steven M. Seitz,et al.  Shape and materials by example: a photometric stereo approach , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Rama Chellappa,et al.  Appearance Characterization of Linear Lambertian Objects, Generalized Photometric Stereo, and Illumination-Invariant Face Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Alfred M. Bruckstein,et al.  Direct Shape Recovery from Photometric Stereo with Shadows , 2013, 2013 International Conference on 3D Vision.

[12]  Abhishek Dutta,et al.  Minimal image sets for robust spherical gradient photometric stereo , 2010, SIGGRAPH ASIA.

[13]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Adam P. Harrison,et al.  Maximum Likelihood Estimation of Depth Maps Using Photometric Stereo , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Abhijeet Ghosh,et al.  Circularly polarized spherical illumination reflectometry , 2010, SIGGRAPH 2010.

[16]  Kiyoharu Aizawa,et al.  Robust photometric stereo using sparse regression , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Daniel Cremers,et al.  Realistic photometric stereo using partial differential irradiance equation ratios , 2015, Comput. Graph..

[18]  Laurence A. Soderblom,et al.  Modeling crater topography and albedo from monoscopic Viking Orbiter images: 1. Methodology , 1984 .

[19]  N Tsumura,et al.  Medical vision : measurement of skin absolute spectral-reflectance image and the application to component analysis , 2001 .

[20]  C.-C. Jay Kuo,et al.  Shape from Photometric Ratio and Stereo , 1996, J. Vis. Commun. Image Represent..

[21]  Jovan Popović,et al.  Dynamic shape capture using multi-view photometric stereo , 2009, SIGGRAPH 2009.

[22]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[23]  Roberto Cipolla,et al.  Overcoming Shadows in 3-Source Photometric Stereo , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Katsushi Ikeuchi,et al.  Median Photometric Stereo as Applied to the Segonko Tumulus and Museum Objects , 2009, International Journal of Computer Vision.

[26]  Alfred M. Bruckstein,et al.  A Direct Differential Approach to Photometric Stereo with Perspective Viewing , 2014, SIAM J. Imaging Sci..

[27]  Rui Huang,et al.  Shape-from-shading under complex natural illumination , 2011, 2011 18th IEEE International Conference on Image Processing.

[28]  Allan D. Jepson,et al.  Polynomial shape from shading , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Michael Brady,et al.  Integrating stereo and photometric stereo to monitor the development of glaucoma , 1991, Image Vis. Comput..

[30]  Alfred M. Bruckstein,et al.  Near Field Photometric Stereo with Point Light Sources , 2014, SIAM J. Imaging Sci..

[31]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[33]  Tien-Tsin Wong,et al.  Dense Photometric Stereo: A Markov Random Field Approach , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Rama Chellappa,et al.  Symmetric Shape-from-Shading Using Self-ratio Image , 2001, International Journal of Computer Vision.

[35]  Jiuai Sun,et al.  Object surface recovery using a multi-light photometric stereo technique for non-Lambertian surfaces subject to shadows and specularities , 2007, Image Vis. Comput..

[36]  K P HornBerthold,et al.  The variational approach to shape from shading , 1986 .

[37]  Daniel Snow,et al.  Shape and albedo from multiple images using integrability , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Pieter Peers,et al.  Estimating Specular Roughness and Anisotropy from Second Order Spherical Gradient Illumination , 2009, Comput. Graph. Forum.

[39]  Takeshi Shakunaga,et al.  Analysis of photometric factors based on photometric linearization. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[40]  Christian Wöhler,et al.  A general framework for three-dimensional surface reconstruction by self-consistent fusion of shading and shadow features , 2005, Pattern Recognit..

[41]  Ronen Basri,et al.  From Shading to Local Shape , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Alex Pentland.  Linear shape from shading , 2004, International Journal of Computer Vision.

[43]  Michael J. Brooks,et al.  The variational approach to shape from shading , 1986, Comput. Vis. Graph. Image Process..

[44]  Ira Kemelmacher-Shlizerman,et al.  Photometric Stereo with General, Unknown Lighting , 2006, International Journal of Computer Vision.

[45]  Maria Petrou,et al.  The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Stefano Soatto,et al.  A Non-local Approach to Shape from Ambient Shading , 2009, SSVM.

[47]  Ravi Ramamoorthi,et al.  On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Melvyn L. Smith,et al.  The analysis of surface texture using photometric stereo acquisition and gradient space domain mapping , 1999, Image Vis. Comput..

[49]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[50]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[51]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[52]  Berthold K. P. Horn.  Height and gradient from shading , 1989, International Journal of Computer Vision.

[53]  Katsushi Ikeuchi,et al.  Consensus photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[54]  Matthew Harker,et al.  Least squares surface reconstruction from measured gradient fields , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.