One-day outdoor photometric stereo via skylight estimation

We present an outdoor photometric stereo method using images captured in a single day. We simulate a sky hemisphere for each image according to its GPS and timestamp, and parameterize the obtained sky hemisphere into a quadratic skylight and a Gaussian sunlight distribution. Unlike previous works which usually model outdoor illumination as a sum of constant ambient light and a distant point light, our method models natural illumination according to a popular sky model and thus provides sufficient constraints for shape reconstruction from one day images. We generate pixel profiles of uniformly sampled unit vectors for the corresponding time of captures and evaluate them using correlation with the actual pixel profiles. The estimated surface normal is refined by MRF optimization. We have tested our method to recover objects and scenes of various sizes in real-world outdoor daylight.

[1]  Katsushi Ikeuchi,et al.  Camera Spectral Sensitivity and White Balance Estimation from Sky Images , 2013, International Journal of Computer Vision.

[2]  Steven M. Seitz,et al.  The Visual Turing Test for Scene Reconstruction , 2013, 2013 International Conference on 3D Vision.

[3]  Katsushi Ikeuchi,et al.  Refining Outdoor Photometric Stereo Based on Sky Model , 2013, IPSJ Trans. Comput. Vis. Appl..

[4]  Rui Huang,et al.  Shape-from-shading under complex natural illumination , 2011, 2011 18th IEEE International Conference on Image Processing.

[5]  In-So Kweon,et al.  Time-of-Flight Sensor Calibration for a Color and Depth Camera Pair , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Takahiro Okabe,et al.  Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Mark Packer,et al.  Depth of focus. , 2008, Ophthalmology.

[9]  Qing Zhang,et al.  Edge-preserving photometric stereo via depth fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Robert Pless,et al.  Heliometric Stereo: Shape from Sun Position , 2012, ECCV.

[11]  Robert Pless,et al.  Consistent Temporal Variations in Many Outdoor Scenes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  In-So Kweon,et al.  High Quality Shape from a Single RGB-D Image under Uncalibrated Natural Illumination , 2013, 2013 IEEE International Conference on Computer Vision.

[13]  Donald P. Greenberg,et al.  A framework for the experimental comparison of solar and skydome illumination , 2014, ACM Trans. Graph..

[14]  I. Reda,et al.  Solar Position Algorithm for Solar Radiation Applications (Revised) , 2008 .

[15]  Simon Fuhrmann,et al.  Photometric stereo for outdoor webcams , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Peter Shirley,et al.  A practical analytic model for daylight , 1999, SIGGRAPH.

[18]  Yasuyuki Matsushita,et al.  Self-calibrating photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  D. B. Judd,et al.  Spectral Distribution of Typical Daylight as a Function of Correlated Color Temperature , 1964 .

[20]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Wojciech Matusik,et al.  What do color changes reveal about an outdoor scene? , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Robert Pless,et al.  The Episolar Constraint: Monocular Shape from Shadow Correspondence , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Michael Weinmann,et al.  Exploring Material Recognition for Estimating Reflectance and Illumination From a Single Image , 2016, MAM@EGSR.

[24]  Katsushi Ikeuchi,et al.  Radiometric Calibration by Rank Minimization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Murali Subbarao,et al.  Depth from defocus: A spatial domain approach , 1994, International Journal of Computer Vision.

[26]  I. Reda,et al.  Solar position algorithm for solar radiation applications , 2004 .

[27]  Richard Perez,et al.  Sky luminance data validation: Comparison of seven models with four data banks , 1994 .

[28]  Alexei A. Efros,et al.  Estimating the Natural Illumination Conditions from a Single Outdoor Image , 2012, International Journal of Computer Vision.

[29]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.

[30]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[31]  Stephen Lin,et al.  Shading-Based Shape Refinement of RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Rama Chellappa,et al.  Direct Analytical Methods for Solving Poisson Equations in Computer Vision Problems , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Szymon Rusinkiewicz,et al.  Time‐Lapse Photometric Stereo and Applications , 2014, Comput. Graph. Forum.

[35]  J. Michalsky,et al.  All-weather model for sky luminance distribution—Preliminary configuration and validation , 1993 .

[36]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[37]  Berthold K. P. Horn,et al.  Shape from shading , 1989 .

[38]  E. Saff,et al.  Distributing many points on a sphere , 1997 .

[39]  Alexei A. Efros,et al.  Estimating natural illumination from a single outdoor image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[40]  Ko Nishino,et al.  Shape and Reflectance from Natural Illumination , 2012, ECCV.

[41]  Edward H. Adelson,et al.  Shape estimation in natural illumination , 2011, CVPR 2011.