EWNet: An early warning classification framework for smart grid based on local-to-global perception

Abstract Early warning mechanism is crucial for maintaining the security and reliability of the power grid system. It remains to be a difficult task in a smart grid system due to complex environments in practice. In this paper, by considering the lack of vision-based datasets and models for early warning classification, we constructed a large-scale image dataset, namely EWSPG1.0, which contains 12,113 images annotated with five levels of early warnings. Moreover, 104,448 object instances with respect to ten categories of high-risk objects and power gird infrastructure were annotated with labels, bounding boxes and polygon masks. On the other hand, we proposed a local-to-global perception framework for early warning classification, namely EWNet. Specifically, a local patch responsor is trained by using image patches extracted from the training set according to the labeled bounding box information of objects. The capability of recognizing high-risk objects and power grid infrastructure is transferred by loading the trained local patch responsor with frozen weights. Features are then fed into a feature integration module and a global classification module for early warning classification of an entire image. In order to evaluate the proposed framework, we benchmarked the proposed framework on our constructed dataset with 11 state-of-the-art deep convolutional neural networks (CNNs)-based classification models. Experimental results exhibit the effectiveness of our proposed method in terms of Top-1 classification accuracy. They also indicate that vision-based early warning classification remains challengeable under power grid surveillance and needs further study in future work.

[1]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[4]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[6]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[8]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[9]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[10]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Gerardo J. Osório,et al.  Artificial Intelligence for Product Quality Inspection toward Smart Industries: Quality Control of Vehicle Non-Conformities , 2020, 2020 9th International Conference on Industrial Technology and Management (ICITM).

[12]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[13]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[14]  Faruk Kazi,et al.  Neural Network Based Early Warning System for an Emerging Blackout in Smart Grid Power Networks , 2014, ISI.

[15]  Qian Zhang,et al.  An Improved YOLOv2 for Vehicle Detection , 2018, Sensors.

[16]  Francois Miralles,et al.  State-of-the-art review of computer vision for the management of power transmission lines , 2014, Proceedings of the 2014 3rd International Conference on Applied Robotics for the Power Industry.

[17]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Tong Wang,et al.  An Early Warning Model for Damage of Power Grid under Typhoon Disaster , 2019, 2019 IEEE 3rd Conference on Energy Internet and Energy System Integration (EI2).

[19]  Laurence T. Yang,et al.  A survey on deep learning for big data , 2018, Inf. Fusion.

[20]  Robert Jenssen,et al.  Automatic autonomous vision-based power line inspection: A review of current status and the potential role of deep learning , 2018, International Journal of Electrical Power & Energy Systems.

[21]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[22]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Mark Sandler,et al.  MobileNetV2: Inverted Residuals and Linear Bottlenecks , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[25]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Abhishek Dutta,et al.  The VIA Annotation Software for Images, Audio and Video , 2019, ACM Multimedia.

[28]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[29]  Yunchao Wei,et al.  Object Proposal Generation With Fully Convolutional Networks , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Shuai Wang,et al.  Engineering Vehicles Detection Based on Modified Faster R-CNN for Power Grid Surveillance , 2018, Sensors.

[32]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[33]  Xiangyu Zhang,et al.  ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design , 2018, ECCV.

[34]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[35]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..

[36]  Matti Pietikäinen,et al.  Deep Learning for Generic Object Detection: A Survey , 2018, International Journal of Computer Vision.

[37]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Nei Kato,et al.  An early warning system against malicious activities for smart grid communications , 2011, IEEE Network.

[39]  Hui Ren,et al.  Early warning signals for critical transitions in power systems , 2015 .

[40]  Hong Wen,et al.  Internet of Things Based Smart Grids Supported by Intelligent Edge Computing , 2019, IEEE Access.

[41]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[42]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[43]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[44]  Ming-Ching Chang,et al.  AI City Challenge 2020 – Computer Vision for Smart Transportation Applications , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[45]  Yimin Yang,et al.  Fusion of transfer learning features and its application in image classification , 2017, 2017 IEEE 30th Canadian Conference on Electrical and Computer Engineering (CCECE).