Efficient Photometric Stereo Using Kernel Regression

Photometric stereo estimates surface normals from multiple images captured under different light directions using a fixed camera. To deal with non-Lambertian reflections, the recent photometric stereo methods employ iterative or optimization frameworks that are computationally expensive. This paper proposes an efficient photometric stereo method using kernel regression, which can be transformed to an eigendecomposition problem. The kernel parameter is variable for each surface point so that it can cope with the variety of general reflectances. The best kernel parameter is automatically determined by leave-one-out cross validation. To improve computational efficiency, the leave-one-out process is accelerated by fast matrix computation and proper normal initialization. The proposed photometric stereo method is extensively evaluated on synthetic and real surfaces with various reflectances. Experimental results validate that the method is computationally efficient and achieves the state-of-the-art accuracy in surface normal estimation.

[1]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[2]  Wojciech Matusik,et al.  Inverse shade trees for non-parametric material representation and editing , 2006, ACM Trans. Graph..

[3]  Kiyoharu Aizawa,et al.  Robust photometric stereo using sparse regression , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Tapan K. Sarkar,et al.  A survey of conjugate gradient algorithms for solution of extreme eigen-problems of a symmetric matrix , 1989, IEEE Trans. Acoust. Speech Signal Process..

[5]  Ravi Ramamoorthi,et al.  On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Long Quan,et al.  The Geometry of Reflectance Symmetries , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  David J. Kriegman,et al.  The Bas-Relief Ambiguity , 2004, International Journal of Computer Vision.

[8]  Zhe Wu,et al.  A Benchmark Dataset and Evaluation for Non-Lambertian and Uncalibrated Photometric Stereo , 2019, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Hui-Liang Shen,et al.  Photometric Stereo for General BRDFs via Reflection Sparsity Modeling , 2015, IEEE Transactions on Image Processing.

[10]  Takeshi Shakunaga,et al.  Analysis of photometric factors based on photometric linearization. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[11]  E. North Coleman,et al.  Obtaining 3-dimensional shape of textured and specular surfaces using four-source photometry , 1982, Comput. Graph. Image Process..

[12]  Jiaya Jia,et al.  Efficient photometric stereo on glossy surfaces with wide specular lobes , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Maria Petrou,et al.  The 4-Source Photometric Stereo Technique for Three-Dimensional Surfaces in the Presence of Highlights and Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[15]  Yongtian Wang,et al.  Robust Photometric Stereo via Low-Rank Matrix Completion and Recovery , 2010, ACCV.

[16]  Katsushi Ikeuchi,et al.  Median Photometric Stereo as Applied to the Segonko Tumulus and Museum Objects , 2009, International Journal of Computer Vision.

[17]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[18]  Qionghai Dai,et al.  Fusing Multiview and Photometric Stereo for 3D Reconstruction under Uncalibrated Illumination , 2011, IEEE Transactions on Visualization and Computer Graphics.

[19]  Steven M. Seitz,et al.  Shape and spatially-varying BRDFs from photometric stereo , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[20]  Takahiro Okabe,et al.  From Intensity Profile to Surface Normal: Photometric Stereo for Unknown Light Sources and Isotropic Reflectances , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  David J. Kriegman,et al.  Resolving the Generalized Bas-Relief Ambiguity by Entropy Minimization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Yasuyuki Matsushita,et al.  Self-calibrating photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Katsushi Ikeuchi,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence Bi-polynomial Modeling of Low-frequency Reflectances , 2022 .

[24]  Yoichi Sato,et al.  Uncalibrated photometric stereo based on elevation angle recovery from BRDF symmetry of isotropic materials , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[26]  Takahiro Okabe,et al.  Attached shadow coding: Estimating surface normals from shadows under unknown reflectance and lighting conditions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  David J. Kriegman,et al.  Toward Reconstructing Surfaces With Arbitrary Isotropic Reflectance : A Stratified Photometric Stereo Approach , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[28]  Gregory J. Ward,et al.  Measuring and modeling anisotropic reflection , 1992, SIGGRAPH.

[29]  Katsushi Ikeuchi,et al.  A biquadratic reflectance model for radiometric image analysis , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Takahiro Okabe,et al.  Shape Reconstruction Based on Similarity in Radiance Changes under Varying Illumination , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[32]  Michael Goesele,et al.  A Survey of Photometric Stereo Techniques , 2015, Found. Trends Comput. Graph. Vis..

[33]  K. Torrance,et al.  Theory for off-specular reflection from roughened surfaces , 1967 .

[34]  Jason Lawrence,et al.  A photometric approach for estimating normals and tangents , 2008, ACM Trans. Graph..

[35]  Sang Wook Lee,et al.  Photometric Stereo from Maximum Feasible Lambertian Reflections , 2010, ECCV.

[36]  Kiyoharu Aizawa,et al.  Photometric Stereo Using Sparse Bayesian Regression for General Diffuse Surfaces , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Tien-Tsin Wong,et al.  Dense Photometric Stereo: A Markov Random Field Approach , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  David J. Kriegman,et al.  Photometric stereo with non-parametric and spatially-varying reflectance , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Katsushi Ikeuchi,et al.  Consensus photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Frédo Durand,et al.  Experimental analysis of BRDF models , 2005, EGSR '05.

[41]  Kiyoharu Aizawa,et al.  Photometric Stereo Using Constrained Bivariate Regression for General Isotropic Surfaces , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .