Detecting Ground Shadows in Outdoor Consumer Photographs

Detecting shadows from images can significantly improve the performance of several vision tasks such as object detection and tracking. Recent approaches have mainly used illumination invariants which can fail severely when the qualities of the images are not very good, as is the case for most consumer-grade photographs, like those on Google or Flickr. We present a practical algorithm to automatically detect shadows cast by objects onto the ground, from a single consumer photograph. Our key hypothesis is that the types of materials constituting the ground in outdoor scenes is relatively limited, most commonly including asphalt, brick, stone, mud, grass, concrete, etc. As a result, the appearances of shadows on the ground are not as widely varying as general shadows and thus, can be learned from a labelled set of images. Our detector consists of a three-tier process including (a) training a decision tree classifier on a set of shadow sensitive features computed around each image edge, (b) a CRF-based optimization to group detected shadow edges to generate coherent shadow contours, and (c) incorporating any existing classifier that is specifically trained to detect grounds in images. Our results demonstrate good detection accuracy (85%) on several challenging images. Since most objects of interest to vision applications (like pedestrians, vehicles, signs) are attached to the ground, we believe that our detector can find wide applicability.

[1]  Yoram Singer,et al.  Logistic Regression, AdaBoost and Bregman Distances , 2000, Machine Learning.

[2]  Erik Reinhard,et al.  Evaluation of color spaces for edge classification in outdoor scenes , 2005, IEEE International Conference on Image Processing 2005.

[3]  Antonio Torralba,et al.  LabelMe: A Database and Web-Based Tool for Image Annotation , 2008, International Journal of Computer Vision.

[4]  Thomas B. Moeslund,et al.  Detection and removal of chromatic moving shadows in surveillance scenarios , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Mark S. Drew,et al.  Detecting Illumination in Images , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[6]  Tai-Pang Wu,et al.  A Bayesian approach for shadow extraction from a single image , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[7]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Cheng Lu,et al.  Entropy Minimization for Shadow Removal , 2009, International Journal of Computer Vision.

[9]  Mads Nielsen,et al.  Computer Vision — ECCV 2002 , 2002, Lecture Notes in Computer Science.

[10]  Dani Lischinski,et al.  The Shadow Meets the Mask: Pyramid‐Based Shadow Removal , 2008, Comput. Graph. Forum.

[11]  Alexei A. Efros,et al.  Recovering Surface Layout from an Image , 2007, International Journal of Computer Vision.

[12]  Graham D. Finlayson,et al.  Simple Shadow Remova , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[13]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[14]  Alexei A. Efros,et al.  Recovering Occlusion Boundaries from a Single Image , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  William T. Freeman,et al.  Learning low-level vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[16]  Alexei A. Efros,et al.  Estimating natural illumination from a single outdoor image , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[17]  Katsushi Ikeuchi,et al.  Illumination from Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Jiandong Tian,et al.  Tricolor Attenuation Model for Shadow Detection , 2009, IEEE Transactions on Image Processing.

[19]  Mark S. Drew,et al.  Removing Shadows from Images , 2002, ECCV.

[20]  Jiří Matas,et al.  Computer Vision - ECCV 2004 , 2004, Lecture Notes in Computer Science.

[21]  Bruce A. Maxwell,et al.  A bi-illuminant dichromatic reflection model for understanding images , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Ayan Chakrabarti,et al.  An Empirical Camera Model for Internet Color Vision , 2009, BMVC.

[23]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Yair Weiss,et al.  Deriving intrinsic images from image sequences , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[25]  Cheng Lu,et al.  Intrinsic Images by Entropy Minimization , 2004, ECCV.

[26]  Edward H. Adelson,et al.  Recovering reflectance and illumination in a world of painted polyhedra , 1993, 1993 (4th) International Conference on Computer Vision.

[27]  Katsushi Ikeuchi,et al.  Illumination normalization with time-dependent intrinsic images for video surveillance , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[28]  Steven J. Gortler,et al.  A perception-based color space for illumination-invariant image processing , 2008, ACM Trans. Graph..

[29]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[30]  G. Finlayson,et al.  Simple Shadow Removal , 2006 .

[31]  Jiejie Zhu,et al.  Learning to recognize shadows in monochromatic natural images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[32]  Graham D. Finlayson,et al.  Shadow Identification using Colour Ratios , 2000, Color Imaging Conference.

[33]  Shree K. Nayar,et al.  A class of photometric invariants: separating material from shape and illumination , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.