Multi-view photometric stereo using surface deformation

This paper presents a hybrid approach for 3D reconstruction by fusing photometric stereo and multi-view stereo. The 3D surface is obtained by capturing a set of images taken from different viewpoints under time-varying illuminations. Key factors in the reconstruction process are surface normals that are obtained from photometric stereo. The surface is initialized by integrating the normals and then refined by performing iterative deformations on the initial surface and thereby optimizing image and normal consistency in multiple views. Benefiting from the employment of the deformation approach, we are able to perform image and normal consistency optimization without using matching windows. Instead, always the complete surface is back-projected. This makes the proposed approach much simpler and more robust compared to window-based approaches, which typically require global optimization with constraints on neighboring windows. Experiments on real-world data and ground-truth data show that for diffuse midsized objects without large depth discontinuities our approach improves the accuracy of the reconstructions compared to exiting approaches.

[1]  Jovan Popović,et al.  Dynamic shape capture using multi-view photometric stereo , 2009, SIGGRAPH 2009.

[2]  Pieter Peers,et al.  Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination , 2007 .

[3]  Jaakko Lehtinen,et al.  Practical SVBRDF capture in the frequency domain , 2013, ACM Trans. Graph..

[4]  Steven M. Seitz,et al.  Shape and Spatially-Varying BRDFs from Photometric Stereo , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Thorsten Thormählen,et al.  Combining photometric normals and multi-view stereo for 3D reconstruction , 2015, CVMP.

[6]  Katsushi Ikeuchi,et al.  A biquadratic reflectance model for radiometric image analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Takahiro Okabe,et al.  Attached shadow coding: Estimating surface normals from shadows under unknown reflectance and lighting conditions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[9]  Jan-Michael Frahm,et al.  PatchMatch Based Joint View Selection and Depthmap Estimation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[11]  Hanspeter Pfister,et al.  Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows , 2010, ECCV.

[12]  Katsushi Ikeuchi,et al.  Elevation Angle from Reflectance Monotonicity: Photometric Stereo for General Isotropic Reflectances , 2012, ECCV.

[13]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  S. Birchfiled.  A Pixel Dissimilarity Measure That Is Insensitive to Image Sampling , 1998 .

[15]  Nahum Kiryati,et al.  Bias Correction in Photometric Stereo using Control Points , 2001, VMV.

[16]  David J. Kriegman,et al.  Reflections on the generalized bas-relief ambiguity , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[17]  Xing Mei,et al.  On building an accurate stereo matching system on graphics hardware , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[18]  Andrew M. Wallace,et al.  Recovering surface reflectance and multiple light locations and intensities from image data , 2008, Pattern Recognit. Lett..

[19]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[20]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[21]  Takeo Kanade,et al.  Determining shape and reflectance of Lambertian, specular, and hybrid surfaces using extended sources , 1989, International Workshop on Industrial Applications of Machine Intelligence and Vision,.

[22]  Roberto Cipolla,et al.  Multiview Photometric Stereo , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Yasuyuki Matsushita,et al.  Multiview Photometric Stereo Using Planar Mesh Parameterization , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Hans-Peter Seidel,et al.  Using Spatially Distributed Patterns for Multiple View Camera Calibration , 2011, MIRAGE.

[26]  Qing Zhang,et al.  Edge-preserving photometric stereo via depth fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Pieter Peers,et al.  Temporal upsampling of performance geometry using photometric alignment , 2010, TOGS.

[28]  Paolo Favaro,et al.  A closed-form solution to uncalibrated photometric stereo via diffuse maxima , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Qionghai Dai,et al.  Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination , 2011, IEEE Transactions on Visualization and Computer Graphics.

[30]  Katsushi Ikeuchi,et al.  Photometric Stereo Using Internet Images , 2014, 2014 2nd International Conference on 3D Vision.

[31]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[32]  Diego F. Nehab,et al.  Efficiently combining positions and normals for precise 3D geometry , 2005, SIGGRAPH 2005.

[33]  Björn Stenger,et al.  Color photometric stereo for multicolored surfaces , 2011, 2011 International Conference on Computer Vision.