Automatic autonomous vision-based power line inspection: A review of current status and the potential role of deep learning

Abstract To maintain the reliability, availability, and sustainability of electricity supply, electricity companies regularly perform visual inspections on their transmission and distribution networks. These inspections have been typically carried out using foot patrol and/or helicopter-assisted methods to plan for necessary repair or replacement works before any major damage, which may cause power outage. This solution is quite slow, expensive, and potentially dangerous. In recent years, numerous researches have been conducted to automate the visual inspections by using automated helicopters, flying robots, and/or climbing robots. However, due to the high accuracy requirements of the task and its unique challenges, automatic vision-based inspection has not been widely adopted. In this paper, with the aim of providing a good starting point for researchers who are interested in developing a fully automatic autonomous vision-based power line inspection system, we conduct an extensive literature review. First, we examine existing power line inspection methods with special attention paid to highlight their advantages and disadvantages. Next, we summarize well-suited tasks and review potential data sources for automatic vision-based inspection. Then, we survey existing automatic vision-based power line inspection systems. Based on that, we propose a new automatic autonomous vision-based power line inspection concept that uses Unmanned Aerial Vehicle (UAV) inspection as the main inspection method, optical images as the primary data source, and deep learning as the backbone of data analysis and inspection. Then, we present an overview of possibilities and challenges of deep vision (deep learning for computer vision) approaches for both UAV navigation and UAV inspection and discuss possible solutions to the challenges. Finally, we conclude the paper with an outlook for the future of this field and propose potential next steps for implementing the concept.

[1]  Linlin Zhu,et al.  A double-side filter based power line recognition method for UAV vision system , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[2]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[5]  Pedro B. Castellucci,et al.  Pole and Crossarm Identification in Distribution Power Line Images , 2013, 2013 Latin American Robotics Symposium and Competition.

[6]  Heiko Neumann,et al.  Fully Convolutional Region Proposal Networks for Multispectral Person Detection , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[7]  I. Golightly,et al.  Modeling and control of a robotic power line inspection vehicle , 2006, 2006 IEEE Conference on Computer Aided Control System Design, 2006 IEEE International Conference on Control Applications, 2006 IEEE International Symposium on Intelligent Control.

[8]  Yan Wang,et al.  DeepContour: A deep convolutional feature learned by positive-sharing loss for contour detection , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[10]  I. Hajnsek,et al.  A tutorial on synthetic aperture radar , 2013, IEEE Geoscience and Remote Sensing Magazine.

[11]  Nabil Aouf,et al.  Power pylon detection and monocular depth estimation from inspection UAVs , 2015, Ind. Robot.

[12]  Farokh B. Bastani,et al.  Robust real-time UAV based power line detection and tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[13]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Przemysław Mazurek,et al.  Application of background estimation and removal techniques for the extraction of the power line components on the digital images for the automatic power line inspection systems , 2008 .

[15]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[16]  Xiaogang Wang,et al.  Factors in Finetuning Deep Model for Object Detection with Long-Tail Distribution , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Hong Luo,et al.  Transmission line icing measurement on photogrammetry method , 2015, International Symposium on Multispectral Image Processing and Pattern Recognition.

[18]  Francois Miralles,et al.  State-of-the-art review of computer vision for the management of power transmission lines , 2014, Proceedings of the 2014 3rd International Conference on Applied Robotics for the Power Industry.

[19]  Juha Hyyppä,et al.  Remote sensing methods for power line corridor surveys , 2016 .

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Antonios Tsourdos,et al.  Development of a fuel cell hybrid-powered unmanned aerial vehicle , 2016, 2016 24th Mediterranean Conference on Control and Automation (MED).

[22]  Ping Li,et al.  Current trends in the development of intelligent unmanned autonomous systems , 2017, Frontiers of Information Technology & Electronic Engineering.

[23]  Rushikesh K. Joshi,et al.  High Level Event Ontology for Multiarea Power System , 2012, IEEE Transactions on Smart Grid.

[24]  Jeremiah Neubert,et al.  DEBC Detection with Deep Learning , 2017, SCIA.

[25]  Dewi I. Jones,et al.  Corner detection and matching for visual tracking during power line inspection , 2003, Image Vis. Comput..

[26]  Xuelong Li,et al.  Power line detection from optical images , 2014, Neurocomputing.

[27]  Lin Wang,et al.  A vision-based method for the broken spacer detection , 2015, 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER).

[28]  Aamir Saeed Malik,et al.  A novel method for vegetation encroachment monitoring of transmission lines using a single 2D camera , 2014, Pattern Analysis and Applications.

[29]  Thomas B. Moeslund,et al.  Thermal cameras and applications: a survey , 2013, Machine Vision and Applications.

[30]  Chongyang Zhang,et al.  A new image-based algorithm for icing detection and icing thickness estimation for Transmission Lines , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[31]  Wu Haibin,et al.  Damper Detection in Helicopter Inspection of Power Transmission Line , 2014, 2014 Fourth International Conference on Instrumentation and Measurement, Computer, Communication and Control.

[32]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Carlos Sampedro,et al.  Towards autonomous detection and tracking of electric towers for aerial power line inspection , 2014, 2014 International Conference on Unmanned Aircraft Systems (ICUAS).

[34]  Shengzhi Du,et al.  Heuristic Bayesian pixel classification for power line inspection , 2010, 2010 3rd International Congress on Image and Signal Processing.

[35]  Anya Castillo,et al.  Risk analysis and management in power outage and restoration: A literature survey , 2014 .

[36]  Lin Lei,et al.  Detecting Insulators in the Image of Overhead Transmission Lines , 2012, ICIC.

[37]  J. M. Lloyd,et al.  Thermal Imaging Systems , 1975 .

[38]  Shengzhi Du,et al.  Hough Transform Tuned Bayesian Classifier for Overhead Power Line Inspection , 2009 .

[39]  J. Radon On the determination of functions from their integral values along certain manifolds , 1986, IEEE Transactions on Medical Imaging.

[40]  J. Koenderink Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[41]  R. Ishino,et al.  Detection system of damaged cables using video obtained from an aerial inspection of transmission lines , 2004, IEEE Power Engineering Society General Meeting, 2004..

[42]  G. Turin,et al.  An introduction to matched filters , 1960, IRE Trans. Inf. Theory.

[43]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[44]  Sansanee Auephanwiriyakul,et al.  Automatic detection of electricity pylons in aerial video sequences , 2010, 2010 International Conference on Electronics and Information Engineering.

[45]  Xiaoxiao Li,et al.  Semantic Image Segmentation via Deep Parsing Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[46]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[47]  Shen-En Chen,et al.  Laser Scanning Technology for Bridge Monitoring , 2012 .

[48]  Aloysius Wehr,et al.  Airborne laser scanning—an introduction and overview , 1999 .

[49]  J. Katrasnik,et al.  A Survey of Mobile Robots for Distribution Power Line Inspection , 2010, IEEE Transactions on Power Delivery.

[50]  Jianbo Shi,et al.  DeepEdge: A multi-scale bifurcated deep network for top-down contour detection , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Wai Ho Li,et al.  Image processing to automate condition assessment of overhead line components , 2010, 2010 1st International Conference on Applied Robotics for the Power Industry.

[52]  Jinhai Cai,et al.  Knowledge-based power line detection for UAV surveillance and inspection systems , 2008, 2008 23rd International Conference Image and Vision Computing New Zealand.

[53]  Tanima Dutta,et al.  Image Analysis-Based Automatic Detection of Transmission Towers using Aerial Imagery , 2015, IbPRIA.

[54]  Qian Wang,et al.  High Speed Automatic Power Line Detection and Tracking for a UAV-Based Inspection , 2012, 2012 International Conference on Industrial Control and Electronics Engineering.

[55]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[56]  Rajeev M Bhujade,et al.  DETECTION OF POWER -LINES IN COMPLEX NATURAL SURROUNDINGS , 2013, ICIT 2013.

[57]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[58]  Aamir Saeed Malik,et al.  Vegetation encroachment monitoring for transmission lines right-of-ways: A survey , 2013 .

[59]  Michael Gerke,et al.  Visual inspection of power lines by U.A.S. , 2014, 2014 International Conference and Exposition on Electrical and Power Engineering (EPE).

[60]  Gorka Sorrosal,et al.  Automatic system for overhead power line inspection using an Unmanned Aerial Vehicle — RELIFO project , 2013, 2013 International Conference on Unmanned Aircraft Systems (ICUAS).

[61]  Sven Behnke,et al.  Multispectral Pedestrian Detection using Deep Fusion Convolutional Neural Networks , 2016, ESANN.

[62]  C. Tao,et al.  Advances in Mobile Mapping Technology , 2009 .

[63]  Silvio Savarese,et al.  Learning Transferrable Representations for Unsupervised Domain Adaptation , 2016, NIPS.

[64]  Bernardino Castillo-Toledo,et al.  Power line inspection via an unmanned aerial system based on the quadrotor helicopter , 2014, MELECON 2014 - 2014 17th IEEE Mediterranean Electrotechnical Conference.

[65]  Flavio Prieto,et al.  Power line detection using a circle based search with UAV images , 2014, 2014 International Conference on Unmanned Aircraft Systems (ICUAS).

[66]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[67]  Bartosz Krawczyk,et al.  Learning from imbalanced data: open challenges and future directions , 2016, Progress in Artificial Intelligence.

[68]  Liu Yan,et al.  Power transmission tower monitoring technology based on TerraSAR-X products , 2011, International Symposium on Lidar and Radar Mapping Technologies.

[69]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[70]  Michael Kampffmeyer,et al.  Semantic Segmentation of Small Objects and Modeling of Uncertainty in Urban Remote Sensing Images Using Deep Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[71]  Janos Toth,et al.  Smart view for a smart grid — Unmanned Aerial Vehicles for transmission lines , 2010, 2010 1st International Conference on Applied Robotics for the Power Industry.

[72]  Youmin Zhang,et al.  Insulator identification from aerial images using Support Vector Machine with background suppression , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).

[73]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[74]  Horst Bischof,et al.  Visual Recognition and Fault Detection for Power Line Insulators , 2014 .

[75]  Xiaolong Wang,et al.  Deep Learning for Polar Bear Detection , 2017, SCIA.

[76]  Jubai An,et al.  A Robust Insulator Detection Algorithm Based on Local Features and Spatial Orders for Aerial Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[77]  Liming Wang,et al.  The Ultraviolet Detection of Corona Discharge in Power Transmission Lines , 2013 .

[78]  Michael I. Jordan,et al.  Unsupervised Domain Adaptation with Residual Transfer Networks , 2016, NIPS.

[79]  Pascual Campoy Cervera,et al.  A supervised approach to electric tower detection and classification for power line inspection , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[80]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[81]  Hang Yin,et al.  Overhead power line detection from UAV video images , 2012, 2012 19th International Conference on Mechatronics and Machine Vision in Practice (M2VIP).