Photometric stereo for strong specular highlights

Photometric stereo is a fundamental technique in computer vision known to produce 3D shape with high accuracy. It uses several input images of a static scene taken from one and the same camera position but under varying illumination. The vast majority of studies in this 3D reconstruction method assume orthographic projection for the camera model. In addition, they mainly use the Lambertian reflectance model as the way that light scatters at surfaces. Thus, providing reliable photometric stereo results from real world objects still remains a challenging task. We address 3D reconstruction by use of a more realistic set of assumptions, combining for the first time the complete Blinn–Phong reflectance model and perspective projection. Furthermore, we compare two different methods of incorporating the perspective projection into our model. Experiments are performed on both synthetic and real world images; the latter do not benefit from laboratory conditions. The results show the high potential of our method even for complex real world applications such as medical endoscopy images which may include many specular highlights.

[1]  Yongtian Wang,et al.  Robust Photometric Stereo via Low-Rank Matrix Completion and Recovery , 2010, ACCV.

[2]  R. Andersen,et al.  Response of MSTd neurons to simulated 3D orientation of rotating planes. , 2002, Journal of neurophysiology.

[3]  Kiriakos N. Kutulakos,et al.  Transparent and Specular Object Reconstruction , 2010, Comput. Graph. Forum.

[4]  William A. P. Smith,et al.  Height from photometric ratio with model-based light source selection , 2016, Comput. Vis. Image Underst..

[5]  Steven M. Seitz,et al.  Shape and spatially-varying BRDFs from photometric stereo , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[6]  Yuji Iwahori,et al.  Shape from Endoscope Image based on Photometric and Geometric Constraints , 2013, KES.

[7]  Michael Breuß,et al.  Making Shape from Shading Work for Real-World Images , 2009, DAGM-Symposium.

[8]  Yehezkel Yeshurun,et al.  Reconstruction of medical images by perspective shape-from-shading , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[9]  Alfred M. Bruckstein,et al.  Two-Image Perspective Photometric Stereo Using Shape-from-Shading , 2012, ACCV.

[10]  Sang Wook Lee,et al.  Photometric Stereo from Maximum Feasible Lambertian Reflections , 2010, ECCV.

[11]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[12]  Michael Breuß,et al.  Fast and accurate surface normal integration on non-rectangular domains , 2016, Computational Visual Media.

[13]  Kenneth Levenberg.  A METHOD FOR THE SOLUTION OF CERTAIN NON – LINEAR PROBLEMS IN LEAST SQUARES , 1944 .

[14]  Yvain Quéau,et al.  Unifying diffuse and specular reflections for the photometric stereo problem , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[15]  Steven M. Seitz,et al.  Shape and materials by example: a photometric stereo approach , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[16]  Michael Breuß,et al.  Perspective photometric stereo beyond Lambert , 2015, International Conference on Quality Control by Artificial Vision.

[17]  Aswin C. Sankaranarayanan,et al.  Specular surface reconstruction from sparse reflection correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Paolo Favaro,et al.  A New Perspective on Uncalibrated Photometric Stereo , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Tommy W. S. Chow,et al.  Shape recovery from shading by a new neural-based reflectance model , 1999, IEEE Trans. Neural Networks.

[20]  J. Payne,et al.  Perspective and form ratio as determinants of relative slant judgments , 1969 .

[21]  Mauricio Galo,et al.  Surface reconstruction using multiple light sources and perspective projection , 1996, Proceedings of 3rd IEEE International Conference on Image Processing.

[22]  Jean-Denis Durou,et al.  Edge-Preserving Integration of a Normal Field: Weighted Least-Squares, TV and L^1 Approaches , 2015, SSVM.

[23]  TankusAriel,et al.  Shape-from-Shading Under Perspective Projection , 2005 .

[24]  Jovan Popovic,et al.  Deformation transfer for triangle meshes , 2004, ACM Trans. Graph..

[25]  S Tibau,et al.  The Role of the Centre of Projection in the Estimation of Slant from Texture of Planar Surfaces , 2001, Perception.

[26]  J. Saunders,et al.  Perception of 3D surface orientation from skew symmetry , 2001, Vision Research.

[27]  Peter Hedman,et al.  Multi-view Reconstruction of Highly Specular Surfaces in Uncontrolled Environments , 2015, 2015 International Conference on 3D Vision.

[28]  Berthold K. P. Horn,et al.  Determining Shape and Reflectance Using Multiple Images , 1978 .

[29]  Ko Nishino,et al.  Multiview Shape and Reflectance from Natural Illumination , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Alessio Del Bue,et al.  Direct Differential Photometric Stereo Shape Recovery of Diffuse and Specular Surfaces , 2016, Journal of Mathematical Imaging and Vision.

[31]  James F. Blinn,et al.  Models of light reflection for computer synthesized pictures , 1977, SIGGRAPH.

[32]  Fabio Camilli,et al.  A Unified Approach to the Well-Posedness of Some Non-Lambertian Models in Shape-from-Shading Theory , 2016, SIAM J. Imaging Sci..

[33]  Steven A. Shafer,et al.  Depth from focusing and defocusing , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Takeshi Shakunaga,et al.  Analysis of photometric factors based on photometric linearization. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[35]  P. Beckmann,et al.  The scattering of electromagnetic waves from rough surfaces , 1963 .

[36]  Katsushi Ikeuchi,et al.  Separating reflection components of textured surfaces using a single image , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  K. A. Stevens.  Surface tilt (the direction of slant): A neglected psychophysical variable , 1983, Perception & psychophysics.

[38]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[39]  Berthold K. P. Horn.  Robot vision , 1986, MIT electrical engineering and computer science series.

[40]  Ari Rosenberg,et al.  The Visual Representation of 3D Object Orientation in Parietal Cortex , 2013, The Journal of Neuroscience.

[41]  Yehezkel Yeshurun,et al.  Shape-from-Shading Under Perspective Projection , 2005, International Journal of Computer Vision.

[42]  Katsushi Ikeuchi,et al.  Median Photometric Stereo as Applied to the Segonko Tumulus and Museum Objects , 2009, International Journal of Computer Vision.

[43]  Bui Tuong Phong.  Illumination for computer generated pictures , 1975, Commun. ACM.

[44]  David J. Kriegman,et al.  Beyond Lambert: reconstructing specular surfaces using color , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[45]  Tien-Tsin Wong,et al.  Dense photometric stereo using tensorial belief propagation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[46]  Christian Wöhler,et al.  3D Computer Vision , 2012 .

[47]  Robert J. Woodham,et al.  Photometric Stereo: A Reflectance Map Technique For Determining Surface Orientation From Image Intensity , 1979, Optics & Photonics.

[48]  Ohad Ben-Shahar,et al.  Shape from Specular Flow , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Greg Humphreys,et al.  Physically Based Rendering: From Theory to Implementation , 2004 .

[50]  Yoshiaki Shirai,et al.  Three-Dimensional Computer Vision , 1987, Symbolic Computation.

[51]  John T. Neu,et al.  Unidirectional Reflectance of Imperfectly Diffuse Surfaces , 1966 .

[52]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[53]  H.D. Tagare,et al.  A framework for the construction of reflectance maps for machine vision , 1993 .

[54]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[55]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[56]  Daniel Cremers,et al.  Realistic photometric stereo using partial differential irradiance equation ratios , 2015, Comput. Graph..

[57]  Hideko F. Norman,et al.  Visual discrimination of local surface structure: Slant, tilt, and curvedness , 2006, Vision Research.

[58]  In-So Kweon,et al.  Specular Reflection Separation Using Dark Channel Prior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[59]  Nahum Kiryati,et al.  Photometric stereo under perspective projection , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.