Estimating natural illumination from a single outdoor image

Given a single outdoor image, we present a method for estimating the likely illumination conditions of the scene. In particular, we compute the probability distribution over the sun position and visibility. The method relies on a combination of weak cues that can be extracted from different portions of the image: the sky, the vertical surfaces, and the ground. While no single cue can reliably estimate illumination by itself, each one can reinforce the others to yield a more robust estimate. This is combined with a data-driven prior computed over a dataset of 6 million Internet photos. We present quantitative results on a webcam dataset with annotated sun positions, as well as qualitative results on consumer-grade photographs downloaded from Internet. Based on the estimated illumination, we show how to realistically insert synthetic 3-D objects into the scene.

[1]  Mark S. Drew,et al.  Diagonal transforms suffice for color constancy , 1993, 1993 (4th) International Conference on Computer Vision.

[2]  J. Michalsky,et al.  All-weather model for sky luminance distribution—Preliminary configuration and validation , 1993 .

[3]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  David W. Jacobs,et al.  In search of illumination invariants , 2001, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[5]  Ronen Basri,et al.  Photometric stereo with general, unknown lighting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[6]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[7]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[8]  Wei Zhang,et al.  Video Compass , 2002, ECCV.

[9]  Katsushi Ikeuchi,et al.  Illumination from Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  I. Reda,et al.  Solar position algorithm for solar radiation applications , 2004 .

[11]  P. Debevec,et al.  Direct HDR capture of the sun and sky , 2004, International Conference on Computer Graphics and Interactive Techniques.

[12]  P. Cavanagh.  The artist as neuroscientist , 2005, Nature.

[13]  Taeone Kim,et al.  A practical single image based approach for estimating illumination distribution from shadows , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[14]  Erik Reinhard,et al.  Evaluation of color spaces for edge classification in outdoor scenes , 2005, IEEE International Conference on Image Processing 2005.

[15]  Tai-Pang Wu,et al.  A Bayesian approach for shadow extraction from a single image , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[16]  Erik Reinhard,et al.  Image-based material editing , 2005, SIGGRAPH '05.

[17]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[19]  David J. Kriegman,et al.  Color Subspaces as Photometric Invariants , 2006, CVPR.

[20]  Alexei A. Efros,et al.  Photo clip art , 2007, ACM Trans. Graph..

[21]  Alexei A. Efros,et al.  What Does the Sky Tell Us about the Camera? , 2008, ECCV.

[22]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[23]  Wojciech Matusik,et al.  What do color changes reveal about an outdoor scene? , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Hassan Foroosh,et al.  Estimating Geo-temporal Location of Stationary Cameras Using Shadow Trajectories , 2008, ECCV.

[25]  Alexei A. Efros,et al.  IM2GPS: estimating geographic information from a single image , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Alexei A. Efros,et al.  What Do the Sun and the Sky Tell Us About the Camera? , 2010, International Journal of Computer Vision.