Multiresolution Fully Convolutional Networks to detect Clouds and Snow through Optical Satellite Images

Clouds and snow have similar spectral features in the visible and near-infrared (VNIR) range and are thus difficult to distinguish from each other in high resolution VNIR images. We address this issue by introducing a shortwave-infrared (SWIR) band where clouds are highly reflective, and snow is absorptive. As SWIR is typically of a lower resolution compared to VNIR, this study proposes a multiresolution fully convolutional neural network (FCN) that can effectively detect clouds and snow in VNIR images. We fuse the multiresolution bands within a deep FCN and perform semantic segmentation at the higher, VNIR resolution. Such a fusion-based classifier, trained in an end-to-end manner, achieved 94.31% overall accuracy and an F1 score of 97.67% for clouds on Resourcesat-2 data captured over the state of Uttarakhand, India. These scores were found to be 30% higher than a Random Forest classifier, and 10% higher than a standalone single-resolution FCN. Apart from being useful for cloud detection purposes, the study also highlights the potential of convolutional neural networks for multi-sensor fusion problems.

[1]  C. Woodcock,et al.  Improvement and expansion of the Fmask algorithm: cloud, cloud shadow, and snow detection for Landsats 4–7, 8, and Sentinel 2 images , 2015 .

[2]  İ. Sönmez,et al.  Snow-covered area determination based on satellite-derived probabilistic snow cover maps , 2016, Arabian Journal of Geosciences.

[3]  Steven D. Miller,et al.  Satellite-Based Imagery Techniques for Daytime Cloud/Snow Delineation from MODIS. , 2005 .

[4]  C. Mätzler,et al.  Possibilities and Limits of Synthetic Aperture Radar for Snow and Glacier Surveying , 1987, Annals of Glaciology.

[5]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[6]  M. Joseph Hughes,et al.  Automated Detection of Cloud and Cloud Shadow in Single-Date Landsat Imagery Using Neural Networks and Spatial Post-Processing , 2014, Remote. Sens..

[7]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Parvaneh Saeedi,et al.  A Cloud Detection Algorithm for Remote Sensing Images Using Fully Convolutional Neural Networks , 2018, 2018 IEEE 20th International Workshop on Multimedia Signal Processing (MMSP).

[9]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Alfred Stein,et al.  Deep Fully Convolutional Networks for the Detection of Informal Settlements in VHR Images , 2017, IEEE Geoscience and Remote Sensing Letters.

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Claudio Persello,et al.  Deep Convolutional Networks for Cloud Detection Using Resourcesat-2 Data , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[13]  Ronald,et al.  Learning representations by backpropagating errors , 2004 .

[14]  Yong Dou,et al.  Airport Detection on Optical Satellite Images Using Deep Convolutional Neural Networks , 2017, IEEE Geoscience and Remote Sensing Letters.

[15]  Jean-Pierre Dedieu,et al.  Annual and Seasonal Glacier-Wide Surface Mass Balance Quantified from Changes in Glacier Surface State: A Review on Existing Methods Using Optical Satellite Imagery , 2017, Remote. Sens..

[16]  Tianqi Chen,et al.  Empirical Evaluation of Rectified Activations in Convolutional Network , 2015, ArXiv.

[17]  Satellite-Based Mapping and Monitoring of Heavy Snowfall in North Western Himalaya and its Hydrologic Consequences , 2017 .

[18]  Zhe Zhu,et al.  Object-based cloud and cloud shadow detection in Landsat imagery , 2012 .

[19]  Albert Rango,et al.  II. Snow hydrology processes and remote sensing , 1993 .

[20]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[21]  Sang-Eun Park,et al.  Dual-Dense Convolution Network for Change Detection of High-Resolution Panchromatic Imagery , 2018, Applied Sciences.

[22]  Nathan Srebro,et al.  The Marginal Value of Adaptive Gradient Methods in Machine Learning , 2017, NIPS.

[23]  Jianping Shi,et al.  Distinguishing Cloud and Snow in Satellite Images via Deep Convolutional Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[24]  Alfred Stein,et al.  Recurrent Multiresolution Convolutional Networks for VHR Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Dennis P. Lettenmaier,et al.  Evaluation of the snow‐covered area data product from MODIS , 2003 .

[26]  Xiaolin Zhu,et al.  An automatic method for screening clouds and cloud shadows in optical satellite image time series in cloudy regions , 2018, Remote Sensing of Environment.