Thermal and Visual Tracking of Photovoltaic Plants for Autonomous UAV inspection

Because photovoltaic (PV) plants require periodic maintenance, using unmanned aerial vehicles (UAV) for inspections can help reduce costs. Usually, the thermal and visual inspection of PV installations works as follows. A UAV equipped with a global positioning system (GPS) receiver is assigned a flight zone, which the UAV will cover back and forth to collect images to be subsequently composed in an orthomosaic. When doing this, the UAV typically flies at a height above the ground that is appropriate to ensure that images overlap even in the presence of GPS positioning errors. However, this approach has two limitations. First, it requires covering the whole flight zone, including “empty” areas between PV module rows. Second, flying high above the ground limits the resolution of the images to be subsequently inspected. The article proposes a novel approach using an autonomous UAV with an RGB and a thermal camera for PV module tracking through segmentation and visual servoing, which does not require a GPS except for measuring the “small” relative displacement between a PV module row and the next one. With this solution, the UAV moves along PV module rows at a lower height than usual and inspects them back and forth in a boustrophedon way by ignoring “empty” areas with no PV modules. Experimental tests performed in simulation and at an actual PV plant are reported, showing a tracking error lower than 0.2 m in most situations when moving at 1.2 m/s.

[1]  Jun Tang,et al.  Dynamic Reallocation Model of Multiple Unmanned Aerial Vehicle Tasks in Emergent Adjustment Scenarios , 2023, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Jing Chen,et al.  Coverage Path Planning Based on the Optimization Strategy of Multiple Solar Powered Unmanned Aerial Vehicles , 2022, Drones.

[3]  Hanqing Wu,et al.  A Method for Estimating On-Field Photovoltaics System Efficiency Using Thermal Imaging and Weather Instrument Data and an Unmanned Aerial Vehicle , 2022, Energies.

[4]  Hicham Hajji,et al.  Developing a deep learning-based layer-3 solution for thermal infrared large-scale photovoltaic module inspection from orthorectified big UAV imagery data , 2022, Int. J. Appl. Earth Obs. Geoinformation.

[5]  Tor Atle Solend,et al.  Modelling the impact of UAV navigation errors on infrared PV inspection data quality and efficiency , 2021, 2021 IEEE 48th Photovoltaic Specialists Conference (PVSC).

[6]  Md Yushalify Misro,et al.  Geometric Modeling Using New Cubic Trigonometric B-Spline Functions with Shape Parameter , 2020, Mathematics.

[7]  H. Ismail,et al.  Enhance PV Panel Detection Using Drone Equipped With RTK , 2020, Volume 7A: Dynamics, Vibration, and Control.

[8]  Soo Young Shin,et al.  Real-time Road Cracks Detection based on Improved Deep Convolutional Neural Network , 2020, 2020 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[9]  Rubén Usamentiaga,et al.  Robust Detection, Classification and Localization of Defects in Large Photovoltaic Plants Based on Unmanned Aerial Vehicles and Infrared Thermography , 2020, Applied Sciences.

[10]  Mohammad Reza Mohammadi,et al.  Automatic Boundary Extraction of Large-Scale Photovoltaic Plants Using a Fully Convolutional Network on Aerial Imagery , 2020, IEEE Journal of Photovoltaics.

[11]  Fausto Pedro García Márquez,et al.  Photovoltaic plant condition monitoring using thermal images analysis by convolutional neural network-based structure , 2020, Renewable Energy.

[12]  Francesco Grimaccia,et al.  A Computer Vision Line-Tracking Algorithm for Automatic UAV Photovoltaic Plants Monitoring Applications , 2020, Energies.

[13]  Jyi-Shane Liu,et al.  UAV System Integration of Real-time Sensing and Flight Task Control for Autonomous Building Inspection Task , 2019, 2019 International Conference on Technologies and Applications of Artificial Intelligence (TAAI).

[14]  Fan Zhang,et al.  OTL-Classifier: Towards Imaging Processing for Future Unmanned Overhead Transmission Line Maintenance , 2019, Electronics.

[15]  Alejandro Barredo Arrieta,et al.  Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI , 2019, Inf. Fusion.

[16]  Mohammadreza Aghaei,et al.  Autonomous Path Planning by Unmanned Aerial Vehicle (UAV) for Precise Monitoring of Large-Scale PV plants , 2019, 2019 IEEE 46th Photovoltaic Specialists Conference (PVSC).

[17]  Antonio Sgorbissa,et al.  Integrated robot planning, path following, and obstacle avoidance in two and three dimensions: Wheeled robots, underwater vehicles, and multicopters , 2019, Int. J. Robotics Res..

[18]  Toon Goedemé,et al.  Fooling Automated Surveillance Cameras: Adversarial Patches to Attack Person Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[19]  Qiang Yang,et al.  Deep Learning Based Module Defect Analysis for Large-Scale Photovoltaic Farms , 2019, IEEE Transactions on Energy Conversion.

[20]  Alessia Saggese,et al.  An intelligent flying system for automatic detection of faults in photovoltaic plants , 2019, J. Ambient Intell. Humaniz. Comput..

[21]  Imane Sebari,et al.  Thermal Infrared and Visual Inspection of Photovoltaic Installations by UAV Photogrammetry—Application Case: Morocco , 2018, Drones.

[22]  Yan Sun,et al.  A Vision-Based Inspection Strategy for Large-Scale Photovoltaic Farms Using an Autonomous UAV , 2018, 2018 17th International Symposium on Distributed Computing and Applications for Business Engineering and Science (DCABES).

[23]  George C. Giakos,et al.  Power Transmission Lines Inspection using Properly Equipped Unmanned Aerial Vehicle (UAV) , 2018, 2018 IEEE International Conference on Imaging Systems and Techniques (IST).

[24]  Johann C. Dauer,et al.  Considerations of Artificial Intelligence Safety Engineering for Unmanned Aircraft , 2018, SAFECOMP Workshops.

[25]  Liping Yang,et al.  Detection of Asphalt Pavement Potholes and Cracks Based on the Unmanned Aerial Vehicle Multispectral Imagery , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[26]  Alessia Saggese,et al.  Multi-Object Tracking by Flying Cameras Based on a Forward-Backward Interaction , 2018, IEEE Access.

[27]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[28]  Imane Sebari,et al.  Inspection of Photovoltaic Installations by Thermo-visual UAV Imagery Application Case: Morocco , 2017, 2017 International Renewable and Sustainable Energy Conference (IRSEC).

[29]  José M. Sebastián,et al.  Passivity based visual servoing of a UAV for tracking crop lines , 2017, 2017 XVII Workshop on Information Processing and Control (RPIC).

[30]  Mohammadreza Aghaei,et al.  Survey on PV Modules’ Common Faults After an O&M Flight Extensive Campaign Over Different Plants in Italy , 2017, IEEE Journal of Photovoltaics.

[31]  John A. Tsanakas,et al.  Advanced inspection of photovoltaic installations by aerial triangulation and terrestrial georeferencing of thermal/visual imagery , 2017 .

[32]  Claudia Buerhop,et al.  Faults and infrared thermographic diagnosis in operating c-Si photovoltaic modules: A review of research and future challenges , 2016 .

[33]  S. Leva,et al.  Image resolution and defects detection in PV inspection by unmanned technologies , 2016, 2016 IEEE Power and Energy Society General Meeting (PESGM).

[34]  Renato Zaccaria,et al.  Visual feedback with multiple cameras in a UAVs Human-Swarm Interface , 2016, Robotics Auton. Syst..

[35]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Philip Jennings,et al.  Detectable faults on recently installed solar modules in Western Australia , 2014 .

[37]  Mohammadreza Aghaei,et al.  Light Unmanned Aerial Vehicles (UAVs) for Cooperative Inspection of PV Plants , 2014, IEEE Journal of Photovoltaics.

[38]  Alvaro García Cazorla,et al.  ROS : Robot Operating System , 2013 .

[39]  Heikki Saari,et al.  Processing and Assessment of Spectrometric, Stereoscopic Imagery Collected Using a Lightweight UAV Spectral Camera for Precision Agriculture , 2013, Remote. Sens..

[40]  Dirk Holz,et al.  Teleoperated Visual Inspection and Surveillance with Unmanned Ground and Aerial Vehicles , 2008, Int. J. Online Eng..

[41]  Jan-Michael Frahm,et al.  A Comparative Analysis of RANSAC Techniques Leading to Adaptive Real-Time Random Sample Consensus , 2008, ECCV.

[42]  M. Veerachary,et al.  Reliability Issues in Photovoltaic Power Processing Systems , 2008, IEEE Transactions on Industrial Electronics.

[43]  Tarek Hamel,et al.  A UAV for bridge inspection: Visual servoing control law with orientation limits , 2007 .

[44]  R. Sengupta,et al.  Vision Based Following of Locally Linear Structures using an Unmanned Aerial Vehicle , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[45]  Renato Zaccaria,et al.  GPS-based localization for a surveillance UGV in outdoor areas , 2005, Proceedings of the Fifth International Workshop on Robot Motion and Control, 2005. RoMoCo '05..

[46]  Raphael A. Finkel What Is an Operating System? , 2004, The Computer Science and Engineering Handbook.

[47]  Maurizio Piaggio,et al.  Autonomous navigation and localization in service mobile robotics , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[48]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[49]  Antonio Bicchi,et al.  Closed loop steering of unicycle like vehicles via Lyapunov techniques , 1995, IEEE Robotics Autom. Mag..

[50]  D. Shashev,et al.  Visual Navigation and Path Tracking Using Street Geometry Information for Image Alignment and Servoing , 2022 .

[51]  Hicham Hajji,et al.  Automatic Extraction of Photovoltaic Panels from UAV Imagery with Object-Based Image Analysis and Machine Learning , 2021, Lecture Notes in Electrical Engineering.

[52]  D. Fridley,et al.  Our Renewable Future: Laying the Path for 100% Clean Energy , 2016 .

[53]  Frederick E. Daum,et al.  Extended Kalman Filters , 2015, Encyclopedia of Systems and Control.

[54]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.