Understanding Rooftop PV Panel Semantic Segmentation of Satellite and Aerial Images for Better Using Machine Learning

Abstract The photovoltaic (PV) industry boom and increased PV applications call for better planning based on accurate and updated data on the installed capacity. Compared with the manual statistical approach, which is often time-consuming and labor-intensive, using satellite/aerial images to estimate the existing PV installed capacity offers a new method with cost-effective and data-consistent features. Previous studies investigated the feasibility of segmenting PV panels from images involving machine learning technologies. However, due to the particular characteristics of PV panel semantic-segmentation, the machine learning tools need to be designed and applied with careful considerations of the issue formulation, data quality, and model explainability. This paper investigated the characteristics of PV panel semantic-segmentation from the perspective of computer vision. The results reveal that the PV panel image data has several specific characteristics: highly class-imbalance and non-concentrated distribution; homogeneous texture and heterogenous color features; and the notable resolution threshold for effective semantic-segmentation. Moreover, this paper provided recommendations for data obtaining and model design, aiming at each observed character from the viewpoints of recent solutions in computer vision, which can be helpful for future improvement of the PV panel semantic-segmentation.

[1]  Vladimir A. Golovko,et al.  Convolutional neural network based solar photovoltaic panel detection in satellite photos , 2017, 2017 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS).

[2]  Larry S. Davis,et al.  An Analysis of Scale Invariance in Object Detection - SNIP , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Julian Togelius,et al.  Procedural Content Generation: From Automatically Generating Game Levels to Increasing Generality in Machine Learning , 2019, ArXiv.

[4]  Rui Zhu,et al.  Assessment of solar photovoltaic potentials on urban noise barriers using street-view imagery , 2021 .

[5]  Abhinav Gupta,et al.  A-Fast-RCNN: Hard Positive Generation via Adversary for Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Alison S. Tomlin,et al.  Methodology for the assessment of PV capacity over a city region using low-resolution LiDAR data and application to the City of Leeds (UK) , 2014 .

[7]  Hamid R. Tizhoosh,et al.  Comparing LBP, HOG and Deep Features for Classification of Histopathology Images , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[8]  Fuchun Sun,et al.  RON: Reverse Connection with Objectness Prior Networks for Object Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[11]  Zhaojian Huang,et al.  Urban solar utilization potential mapping via deep learning technology: A case study of Wuhan, China , 2019, Applied Energy.

[12]  M. Obersteiner,et al.  1.6 Million transactions replicate distributed PV market slowdown by COVID-19 lockdown , 2021, Applied energy.

[13]  Yongtu Liang,et al.  A voyage with minimal fuel consumption for cruise ships , 2019, Journal of Cleaner Production.

[14]  Jin Yang,et al.  Analysis of Distributed Photovoltaic Financing : A Case Study Approach of Crowd-funding with Photovoltaic Water Pumping System in Microgrids , 2016 .

[15]  Julieta Schallenberg-Rodriguez,et al.  Photovoltaic techno-economical potential on roofs in regions and islands: The case of the Canary Islands. Methodological review and methodology proposal , 2013 .

[16]  Subhransu Maji,et al.  DeepRoof: A Data-driven Approach For Solar Potential Estimation Using Rooftop Imagery , 2019, KDD.

[17]  Zhao Zhen,et al.  Deep Learning Based Surface Irradiance Mapping Model for Solar PV Power Forecasting Using Sky Image , 2020, IEEE Transactions on Industry Applications.

[18]  Jinyue Yan,et al.  City-level analysis of subsidy-free solar photovoltaic electricity price, profits and grid parity in China , 2019, Nature Energy.

[19]  Jordan M. Malof,et al.  Automatic solar photovoltaic panel detection in satellite imagery , 2015, 2015 International Conference on Renewable Energy Research and Applications (ICRERA).

[20]  S. Sergyan,et al.  Color histogram features based image classification in content-based image retrieval systems , 2008, 2008 6th International Symposium on Applied Machine Intelligence and Informatics.

[21]  Enhong Chen,et al.  Is Sampling Heuristics Necessary in Training Deep Object Detectors , 2019 .

[22]  Larry S. Davis,et al.  SNIPER: Efficient Multi-Scale Training , 2018, NeurIPS.

[23]  Kaiming He,et al.  Panoptic Feature Pyramid Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Jinyue Yan,et al.  Economic optimization of photovoltaic water pumping systems for irrigation , 2015 .

[25]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[26]  Pietro Perona,et al.  Pedestrian Detection: An Evaluation of the State of the Art , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Pierre Alliez,et al.  Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[28]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  A. Suruliandi,et al.  Texture and color feature extraction for classification of melanoma using SVM , 2016, 2016 International Conference on Computing Technologies and Intelligent Data Engineering (ICCTIDE'16).

[30]  Jordan M. Malof,et al.  Deep Convolutional Segmentation of Remote Sensing Imagery: A Simple and Efficient Alternative to Stitching Output Labels , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[31]  Chiman Kwan,et al.  Blind Quality Assessment of Fused WorldView-3 Images by Using the Combinations of Pansharpening and Hypersharpening Paradigms , 2017, IEEE Geoscience and Remote Sensing Letters.

[32]  Alison S. Tomlin,et al.  Modelling of roof geometries from low-resolution LiDAR data for city-scale solar energy applications using a neighbouring buildings method , 2015 .

[33]  Long Wang,et al.  The automatic segmentation of residential solar panels based on satellite images: A cross learning driven U-Net method , 2020, Appl. Soft Comput..

[34]  Ian McCallum,et al.  Suitable and optimal locations for implementing photovoltaic water pumping systems for grassland irrigation in China , 2017 .

[35]  Sinan Kalkan,et al.  Imbalance Problems in Object Detection: A Review , 2020, IEEE transactions on pattern analysis and machine intelligence.

[36]  Jordan M. Malof,et al.  Distributed solar photovoltaic array location and extent dataset for remote sensing object identification , 2016, Scientific Data.

[37]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[38]  K. Sri Rama Krishna,et al.  Content based Image Retrieval based on Different Global and Local Color Histogram Methods: A Survey , 2017 .

[39]  Jiangye Yuan,et al.  2016 Ieee International Conference on Big Data (big Data) Large-scale Solar Panel Mapping from Aerial Images Using Deep Convolutional Networks , 2022 .

[40]  Uwe Stilla,et al.  Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection , 2016, ISPRS Journal of Photogrammetry and Remote Sensing.

[41]  Jamie Sherrah,et al.  Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery , 2016, ArXiv.

[42]  Bikash Joshi,et al.  Rooftop Detection for Planning of Solar PV Deployment: A Case Study in Abu Dhabi , 2014, DARE.

[43]  G. Štumberger,et al.  Rating of roofs’ surfaces regarding their solar potential and suitability for PV systems, based on LiDAR data , 2013 .

[44]  Jiafan Yu,et al.  DeepSolar: A Machine Learning Framework to Efficiently Construct a Solar Deployment Database in the United States , 2018, Joule.

[45]  Jinyue Yan,et al.  Techno-economic assessment of photovoltaic power generation mounted on cooling towers , 2021, Energy Conversion and Management.

[46]  C. Waelbroeck,et al.  Consistently dated Atlantic sediment cores over the last 40 thousand years , 2019, Scientific Data.

[47]  D. Assouline,et al.  Quantifying rooftop photovoltaic solar energy potential: A machine learning approach , 2017 .

[48]  Jinyue Yan,et al.  Potential analysis of roof-mounted solar photovoltaics in Sweden , 2020, Applied Energy.

[49]  Jordan M. Malof,et al.  Application of a semantic segmentation convolutional neural network for accurate automatic detection and mapping of solar photovoltaic arrays in aerial imagery , 2018, ArXiv.

[50]  Shuicheng Yan,et al.  Clothes Co-Parsing Via Joint Image Segmentation and Labeling With Application to Clothing Retrieval , 2016, IEEE Transactions on Multimedia.

[51]  N. Fueyo,et al.  A method for estimating the geographical distribution of the available roof surface area for large-scale photovoltaic energy-potential evaluations , 2008 .

[52]  Bo Li,et al.  An Improving Technique of Color Histogram in Segmentation-based Image Retrieval , 2009, 2009 Fifth International Conference on Information Assurance and Security.

[53]  Christina M. Kennedy,et al.  Mapping global development potential for renewable energy, fossil fuels, mining and agriculture sectors , 2019, Scientific Data.

[54]  Xin Yi,et al.  LBP-Based Segmentation of Defocus Blur , 2016, IEEE Transactions on Image Processing.

[55]  Vladimir Iglovikov,et al.  Satellite Imagery Feature Detection using Deep Convolutional Neural Network: A Kaggle Competition , 2017, ArXiv.

[56]  Rui Han,et al.  A New Feature Pyramid Network for Object Detection , 2019, 2019 International Conference on Virtual Reality and Intelligent Systems (ICVRIS).

[57]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[58]  W. Fichtner,et al.  A high-resolution determination of the technical potential for residential-roof-mounted photovoltaic systems in Germany , 2014 .

[59]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[60]  Wesley J. Cole,et al.  The evolving energy and capacity values of utility-scale PV-plus-battery hybrid system architectures , 2021, Advances in Applied Energy.

[61]  Meng Lu,et al.  A scale robust convolutional neural network for automatic building extraction from aerial and satellite imagery , 2018, International Journal of Remote Sensing.

[62]  Carmen Gervet,et al.  An integrated GIS and robust optimization framework for solar PV plant planning scenarios at utility scale , 2020, Applied Energy.

[63]  Dong ping Tian,et al.  A Review on Image Feature Extraction and Representation Techniques , 2013 .

[64]  Joseph Walsh,et al.  Real-time monitoring of powder blend composition using near infrared spectroscopy , 2017, 2017 Eleventh International Conference on Sensing Technology (ICST).

[65]  Jordan M. Malof,et al.  Large-Scale Semantic Classification: Outcome of the First Year of Inria Aerial Image Labeling Benchmark , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[66]  Delphine Bresch-Pietri,et al.  Optimizing the self-consumption of residential photovoltaic energy and quantification of the impact of production forecast uncertainties , 2021 .

[67]  T. Landelius,et al.  A gridded optimization model for photovoltaic applications , 2020 .

[68]  Sebastian Ramos,et al.  The Cityscapes Dataset for Semantic Urban Scene Understanding , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  T. Santos,et al.  Photovoltaic potential in a Lisbon suburb using LiDAR data , 2012 .