One-Day Outdoor Photometric Stereo Using Skylight Estimation

We present an outdoor photometric stereo method using images captured in a single day. We simulate a sky hemisphere for each image according to its GPS and timestamp and parameterize the obtained sky hemisphere into quadratic skylight and Gaussian sunlight distributions. Our previous work recovered an outdoor scene on a clear day, whereas the current paper shows that cloudy days can provide better illumination conditions for surface orientation recovery, and hence we propose a modified sky model to represent a well-conditioned skylight distribution for outdoor photometric stereo. The proposed method models natural illumination according to a sky model, providing sufficient constraints for shape reconstruction from 1-day images. We tested the proposed method to recover various sized objects and scenes from real-world outdoor daylight images and verified the method using synthetic and real data experiments.

[1]  Yannick Hold-Geoffroy,et al.  What Is a Good Day for Outdoor Photometric Stereo? , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[2]  Rui Huang,et al.  Shape-from-shading under complex natural illumination , 2011, 2011 18th IEEE International Conference on Image Processing.

[3]  In-So Kweon,et al.  Time-of-Flight Sensor Calibration for a Color and Depth Camera Pair , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[5]  Steven M. Seitz,et al.  The Visual Turing Test for Scene Reconstruction , 2013, 2013 International Conference on 3D Vision.

[6]  Szymon Rusinkiewicz,et al.  Time‐Lapse Photometric Stereo and Applications , 2014, Comput. Graph. Forum.

[7]  Qing Zhang,et al.  Edge-preserving photometric stereo via depth fusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Takahiro Okabe,et al.  Uncalibrated Photometric Stereo for Unknown Isotropic Reflectances , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Yasuyuki Matsushita,et al.  Photometric Stereo in the Wild , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[10]  Robert Pless,et al.  Consistent Temporal Variations in Many Outdoor Scenes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Yasuyuki Matsushita,et al.  Self-calibrating photometric stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Wojciech Matusik,et al.  What do color changes reveal about an outdoor scene? , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.

[14]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Katsushi Ikeuchi,et al.  Radiometric Calibration by Rank Minimization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Robert Pless,et al.  Heliometric Stereo: Shape from Sun Position , 2012, ECCV.

[17]  Katsushi Ikeuchi,et al.  Camera Spectral Sensitivity and White Balance Estimation from Sky Images , 2013, International Journal of Computer Vision.

[18]  Rama Chellappa,et al.  Direct Analytical Methods for Solving Poisson Equations in Computer Vision Problems , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Katsushi Ikeuchi,et al.  Refining Outdoor Photometric Stereo Based on Sky Model , 2013, IPSJ Trans. Comput. Vis. Appl..

[20]  Yannick Hold-Geoffroy,et al.  x-Hour Outdoor Photometric Stereo , 2015, 2015 International Conference on 3D Vision.

[21]  Alexander Wilkie,et al.  Adding a Solar-Radiance Function to the Hošek-Wilkie Skylight Model , 2013, IEEE Computer Graphics and Applications.

[22]  J. Koenderink,et al.  Surface perception in pictures , 1992, Perception & psychophysics.

[23]  In-So Kweon,et al.  High Quality Shape from a Single RGB-D Image under Uncalibrated Natural Illumination , 2013, 2013 IEEE International Conference on Computer Vision.

[24]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[26]  Alexei A. Efros,et al.  Estimating natural illumination from a single outdoor image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[27]  Wojciech Matusik,et al.  Factored time-lapse video , 2007, ACM Trans. Graph..

[28]  Ko Nishino,et al.  Shape and Reflectance from Natural Illumination , 2012, ECCV.

[29]  In-So Kweon,et al.  One-day outdoor photometric stereo via skylight estimation , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Edward H. Adelson,et al.  Shape estimation in natural illumination , 2011, CVPR 2011.

[31]  Yannick Hold-Geoffroy,et al.  Deep Outdoor Illumination Estimation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  P. Grossmann,et al.  Depth from focus , 1987, Pattern Recognit. Lett..

[33]  Stephen Lin,et al.  Shading-Based Shape Refinement of RGB-D Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Murali Subbarao,et al.  Depth from defocus: A spatial domain approach , 1994, International Journal of Computer Vision.

[35]  Ko Nishino,et al.  Reflectance and Natural Illumination from a Single Image , 2012, ECCV.

[36]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  P. Hanrahan,et al.  On the relationship between radiance and irradiance: determining the illumination from images of a convex Lambertian object. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[38]  Alexei A. Efros,et al.  Estimating the Natural Illumination Conditions from a Single Outdoor Image , 2012, International Journal of Computer Vision.

[39]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[40]  E. Saff,et al.  Distributing many points on a sphere , 1997 .

[41]  Berthold K. P. Horn.  Obtaining shape from shading information , 1989 .

[42]  정지영,et al.  Outdoor photometric stereo for 3D scene recovery = 야외 환경의 3차원 복원을 위한 광도 스테레오 , 2016 .

[43]  Simon Fuhrmann,et al.  Photometric stereo for outdoor webcams , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Marc Levoy,et al.  Zippered polygon meshes from range images , 1994, SIGGRAPH.

[45]  Peter Shirley,et al.  A practical analytic model for daylight , 1999, SIGGRAPH.

[46]  Robert Pless,et al.  The Episolar Constraint: Monocular Shape from Shadow Correspondence , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.