The Episolar Constraint: Monocular Shape from Shadow Correspondence

Shadows encode a powerful geometric cue: if one pixel casts a shadow onto another, then the two pixels are colinear with the lighting direction. Given many images over many lighting directions, this constraint can be leveraged to recover the depth of a scene from a single viewpoint. For outdoor scenes with solar illumination, we term this the episolar constraint, which provides a convex optimization to solve for the sparse depth of a scene from shadow correspondences, a method to reduce the search space when finding shadow correspondences, and a method to geometrically calibrate a camera using shadow constraints. Our method constructs a dense network of nonlocal constraints which complements recent work on outdoor photometric stereo and cloud based cues for 3D. We demonstrate results across a variety of time-lapse sequences from web cams "in the wild".

[1]  Hassan Foroosh,et al.  Using solar shadow trajectories for camera calibration , 2008, 2008 15th IEEE International Conference on Image Processing.

[2]  Yongtian Wang,et al.  Robust Photometric Stereo via Low-Rank Matrix Completion and Recovery , 2010, ACCV.

[3]  Robert Pless,et al.  Heliometric Stereo: Shape from Sun Position , 2012, ECCV.

[4]  Ryo Furukawa,et al.  Shape Reconstruction and Camera Self-Calibration Using Cast Shadows and Scene Geometries , 2009, International Journal of Computer Vision.

[5]  Tai-Pang Wu,et al.  Dense Photometric Stereo by Expectation Maximization , 2006, ECCV.

[6]  Xiaochun Cao,et al.  Camera calibration and light source orientation from solar shadows , 2007, Comput. Vis. Image Underst..

[7]  D. Marquardt.  An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[8]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[9]  I. Reda,et al.  Solar position algorithm for solar radiation applications , 2004 .

[10]  John R. Kender,et al.  An optimal algorithm for the derivation of shape from shadows , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[12]  Robert Pless,et al.  Webcam geo-localization using aggregate light levels , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[13]  Robert Pless,et al.  Using cloud shadows to infer scene structure and camera calibration , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Pietro Perona,et al.  Shadow Carving , 2001, ICCV.

[15]  Robert Pless,et al.  Consistent Temporal Variations in Many Outdoor Scenes , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Robert Pless,et al.  Adventures in archiving and using three years of webcam images , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[17]  Simon Fuhrmann,et al.  Photometric stereo for outdoor webcams , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  David J. Kriegman,et al.  ShadowCuts: Photometric Stereo with Shadows , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Yizhou Yu,et al.  Shadow Graphs and Surface Reconstruction , 2002, ECCV.

[20]  Kenneth A. Loparo,et al.  Reconstruction of three-dimensional surfaces from two-dimensional binary images , 1989, IEEE Trans. Robotics Autom..

[21]  Xiaochun Cao,et al.  Camera calibration and geolocation estimation from two shadow trajectories , 2010 .

[22]  Hassan Foroosh,et al.  Estimating Geo-temporal Location of Stationary Cameras Using Shadow Trajectories , 2008, ECCV.

[23]  Scott F. Page,et al.  A method for 3D scene recognition using shadow information and a single fixed viewpoint , 2012, Defense + Commercial Sensing.

[24]  Wojciech Matusik,et al.  Factored time-lapse video , 2007, ACM Trans. Graph..

[25]  Derek Hoiem,et al.  Single-image shadow detection and removal using paired regions , 2011, CVPR 2011.

[26]  Hanspeter Pfister,et al.  Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows , 2010, ECCV.

[27]  Takeo Kanade,et al.  Using shadows in finding surface orientations , 1983, Comput. Vis. Graph. Image Process..

[28]  Shree K. Nayar,et al.  What is the space of camera response functions? , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[29]  Robert Pless,et al.  Webcams in context: web interfaces to create live 3D environments , 2010, ACM Multimedia.

[30]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.

[31]  Bogdan Raducanu,et al.  Synthetic ground truth dataset to detect shadows cast by static objects in outdoors , 2012, VIGTA '12.

[32]  Gregory Dudek,et al.  On 3-D surface reconstruction using shape from shadows , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[33]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Thomas O. Binford,et al.  The Interpretation of Three-Dimensional Structure from Image Curves , 1981, IJCAI.