InNet: Learning to Detect Shadows with Injection Network

Shadows bring great challenges but also play essential roles in image understanding. Most recent shadow detection methods are based on patches, then further reasoning method is required for the obtaining of a completed shadow detection result for an image. In this paper, an injection network is proposed to detect shadow regions for the whole image directly. In order to maintain as many as details, the skip structure is applied to directly inject the details from convolutional layers to de-convolutional layers. Meanwhile, a weighted loss function is proposed for the network training. With this adapted loss function, the network becomes more sensitive to errors of shadow regions. Thus the proposed network can focus on the learning of robust shadow features. Furthermore, a shadow refinement method is proposed to optimize the boundary region of shadows. In the experiments, the proposed methods are extensively evaluated on two popular datasets and shown better performance on shadow detection compared with current methods.

[1]  Alexei A. Efros,et al.  Detecting Ground Shadows in Outdoor Consumer Photographs , 2010, ECCV.

[2]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[3]  Xiaoyue Jiang,et al.  Shadow Detection based on Colour Segmentation and Estimated Illumination , 2011, BMVC.

[4]  Dimitris Samaras,et al.  Leave-One-Out Kernel Optimization for Shadow Detection , 2018, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Xiaoyue Jiang,et al.  Correlation-Based Intrinsic Image Extraction from a Single Image , 2010, ECCV.

[8]  Karianto Leman,et al.  Shadow optimization from structured deep edge detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  H KhanSalman,et al.  Automatic Shadow Detection and Removal from a Single Image , 2016 .

[10]  Cheng Lu,et al.  On the removal of shadows from images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Dimitris Samaras,et al.  Shadow Detection with Conditional Generative Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Joost van de Weijer,et al.  Describing Reflectances for Color Segmentation Robust to Shadows, Highlights, and Textures , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Brian C. Lovell,et al.  Shadow detection: A survey and comparative evaluation of recent methods , 2012, Pattern Recognit..

[14]  Katsushi Ikeuchi,et al.  Illumination from Shadows , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Mohammed Bennamoun,et al.  Automatic Shadow Detection and Removal from a Single Image , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Derek Hoiem,et al.  Single-image shadow detection and removal using paired regions , 2011, CVPR 2011.

[17]  Jiejie Zhu,et al.  Learning to recognize shadows in monochromatic natural images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Larry D. Hostetler,et al.  The estimation of the gradient of a density function, with applications in pattern recognition , 1975, IEEE Trans. Inf. Theory.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Zhaoqiang Xia,et al.  Multiorientation scene text detection via coarse-to-fine supervision-based convolutional networks , 2018, J. Electronic Imaging.

[22]  Dimitris Samaras,et al.  Large-Scale Training of Shadow Detectors with Noisily-Annotated Shadow Examples , 2016, ECCV.

[23]  Takahiro Okabe,et al.  Attached shadow coding: Estimating surface normals from shadows under unknown reflectance and lighting conditions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[24]  Jiandong Tian,et al.  New spectrum ratio properties and features for shadow detection , 2016, Pattern Recognit..

[25]  Michael Werman,et al.  Vertical Parallax from Moving Shadows , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Dimitris Samaras,et al.  Single Image Shadow Detection Using Multiple Cues in a Supermodular MRF , 2013, BMVC.

[27]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Nijad Al-Najdawi,et al.  A survey of cast shadow detection algorithms , 2012, Pattern Recognit. Lett..

[29]  Mohammed Bennamoun,et al.  Automatic Feature Learning for Robust Shadow Detection , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.