Evaluation of Deep Learning-Based Neural Network Methods for Cloud Detection and Segmentation

This paper presents a systematic approach for accurate short-time cloud coverage prediction based on a machine learning (ML) approach. Based on a newly built omnidirectional ground-based sky camera system, local training and evaluation data sets were created. These were used to train several state-of-the-art deep neural networks for object detection and segmentation. For this purpose, the camera-generated a full hemispherical image every 30 min over two months in daylight conditions with a fish-eye lens. From this data set, a subset of images was selected for training and evaluation according to various criteria. Deep neural networks, based on the two-stage R-CNN architecture, were trained and compared with a U-net segmentation approach implemented by CloudSegNet. All chosen deep networks were then evaluated and compared according to the local situation.

[1]  Nicolas Usunier,et al.  End-to-End Object Detection with Transformers , 2020, ECCV.

[2]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[3]  C. W. Chow,et al.  A method for cloud detection and opacity classification based on ground based sky imagery , 2012 .

[4]  Minsu Kim,et al.  A Study of Developing a Prediction Equation of Electricity Energy Output via Photovoltaic Modules , 2021, Energies.

[5]  Anton H. Westveld,et al.  Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation , 2005 .

[6]  J. Kleissl,et al.  Intra-hour forecasting with a total sky imager at the UC San Diego solar energy testbed , 2011 .

[7]  Stefan Winkler,et al.  CloudSegNet: A Deep Network for Nychthemeron Cloud Image Segmentation , 2019, IEEE Geoscience and Remote Sensing Letters.

[8]  Ning Xu,et al.  Deep Image Matting , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Carlos F.M. Coimbra,et al.  Forecasting of Global Horizontal Irradiance Using Sky Cover Indices , 2013 .

[10]  Maurice Schmeits,et al.  Comparison of statistical post-processing methods for probabilistic NWP forecasts of solar radiation , 2019, Solar Energy.

[11]  A. Raftery,et al.  Using Bayesian Model Averaging to Calibrate Forecast Ensembles , 2005 .

[12]  Christopher A. T. Ferro,et al.  A comparison of ensemble post‐processing methods for extreme events , 2014 .

[13]  H. Beyer,et al.  Solar energy assessment using remote sensing technologies , 2003 .

[14]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Zhiwei Li,et al.  CLOUD DETECTION BY FUSING MULTI-SCALE CONVOLUTIONAL FEATURES , 2018 .

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  A Cazorla,et al.  Development of a sky imager for cloud cover assessment. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  Takeyoshi Kato,et al.  Chapter 4 – Prediction of photovoltaic power generation output and network operation , 2016 .

[19]  Thomas M. Hamill,et al.  Probabilistic Precipitation Forecast Postprocessing Using Quantile Mapping and Rank-Weighted Best-Member Dressing , 2018, Monthly Weather Review.

[20]  Hsu-Yung Cheng,et al.  Cloud detection in all-sky images via multi-scale neighborhood features and multiple supervised learning techniques , 2016 .

[21]  Domonkos Varga,et al.  Multi-pooled Inception features for no-reference image quality assessment , 2020, Applied Sciences.

[22]  Hans-Georg Beyer,et al.  Irradiance Forecasting for the Power Prediction of Grid-Connected Photovoltaic Systems , 2009, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[23]  Xinyue Su,et al.  Prediction of Short-Time Cloud Motion Using a Deep-Learning Model , 2020, Atmosphere.

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[26]  Stephan Rasp,et al.  Neural networks for post-processing ensemble weather forecasts , 2018, Monthly Weather Review.

[27]  Olivier Mestre,et al.  Calibrated Ensemble Forecasts Using Quantile Regression Forests and Ensemble Model Output Statistics , 2016 .

[28]  B. Espinar,et al.  Surface solar irradiance estimation with low-cost fish-eye camera , 2012 .

[29]  Tao Xiang,et al.  Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers , 2020, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[31]  Long Chen,et al.  Dense crowd counting based on perspective weight model using a fisheye camera , 2015 .

[32]  T. Hoff,et al.  Validation of short and medium term operational solar radiation forecasts in the US , 2010 .