Cooperative coverage path planning for visual inspection

Abstract This article addresses the inspection problem of a complex 3D infrastructure using multiple Unmanned Aerial Vehicles (UAVs). The main novelty of the proposed scheme stems from the establishment of a theoretical framework capable of providing a path for accomplishing a full coverage of the infrastructure, without any further simplifications (number of considered representation points), by slicing it by horizontal planes to identify branches and assign specific areas to each agent as a solution to an overall optimization problem. Furthermore, the image streams collected during the coverage task are post-processed using Structure from Motion, stereo SLAM and mesh reconstruction algorithms, while the resulting 3D mesh can be used for further visual inspection purposes. The performance of the proposed Collaborative-Coverage Path Planning (C-CPP) has been experimentally evaluated in multiple indoor and realistic outdoor infrastructure inspection experiments and as such it is also contributing significantly towards real life applications for UAVs.

[1]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[2]  R. Siegwart,et al.  Aerial service robots for visual inspection of thermal power plant boiler systems , 2012, 2012 2nd International Conference on Applied Robotics for the Power Industry (CARPI).

[3]  Jianliang Tang,et al.  Complete Solution Classification for the Perspective-Three-Point Problem , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Stephen J. Wright,et al.  Sequential Quadratic Programming , 1999 .

[5]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[6]  François Michaud,et al.  Memory management for real-time appearance-based loop closure detection , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Roland Siegwart,et al.  Three-dimensional coverage path planning via viewpoint resampling and tour optimization for aerial robots , 2015, Autonomous Robots.

[8]  Tarek Hamel,et al.  A UAV for bridge inspection: Visual servoing control law with orientation limits , 2007 .

[9]  Howie Choset,et al.  Exact Cellular Decomposition of Closed Orientable Surfaces Embedded in R3. , 2001 .

[10]  Marc Carreras,et al.  Planning coverage paths on bathymetric maps for in-detail inspection of the ocean floor , 2013, 2013 IEEE International Conference on Robotics and Automation.

[11]  Wolfram Burgard,et al.  An evaluation of the RGB-D SLAM system , 2012, 2012 IEEE International Conference on Robotics and Automation.

[12]  Marc Carreras,et al.  A survey on coverage path planning for robotics , 2013, Robotics Auton. Syst..

[13]  Pierre Hansen,et al.  Cluster analysis and mathematical programming , 1997, Math. Program..

[14]  Anthony Tzes,et al.  Switching model predictive attitude control for a quadrotor helicopter subject to atmospheric disturbances , 2011 .

[15]  Karl Henrik Johansson,et al.  Cooperative coverage for surveillance of 3D structures , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Roland Siegwart,et al.  Robust visual inertial odometry using a direct EKF-based approach , 2015, IROS 2015.

[17]  Vijay Kumar,et al.  Time-optimal UAV trajectory planning for 3D urban structure coverage , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Narcís Palomeras,et al.  Coverage Path Planning with Real‐time Replanning and Surface Reconstruction for Inspection of Three‐dimensional Underwater Structures using Autonomous Underwater Vehicles , 2015, J. Field Robotics.

[19]  Pere Ridao,et al.  Coverage path planning with realtime replanning for inspection of 3D underwater structures , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Roland Siegwart,et al.  Model Predictive Control for Trajectory Tracking of Unmanned Aerial Vehicles Using Robot Operating System , 2017 .

[21]  F. Sibel Salman,et al.  A mixed-integer programming approach to the clustering problem with an application in customer segmentation , 2006, Eur. J. Oper. Res..

[22]  Antonio Barrientos,et al.  Aerial remote sensing in agriculture: A practical approach to area coverage and path planning for fleets of mini aerial robots , 2011, J. Field Robotics.

[23]  Anthony Tzes,et al.  Model predictive quadrotor control: attitude, altitude and position experimental studies , 2012 .

[24]  Jun-Hai Yong,et al.  An offset algorithm for polyline curves , 2007, Comput. Ind..

[25]  Keinosuke Fukunaga,et al.  A Branch and Bound Clustering Algorithm , 1975, IEEE Transactions on Computers.

[26]  Laurence A. Wolsey,et al.  Mixed Integer Programming , 2008, Wiley Encyclopedia of Computer Science and Engineering.

[27]  George Nikolakopoulos,et al.  Coordination of Helicopter UAVs for Aerial Forest-Fire Surveillance , 2009 .

[28]  Andrew Zisserman,et al.  Efficient Visual Search of Videos Cast as Text Retrieval , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  David P. Dobkin,et al.  The quickhull algorithm for convex hulls , 1996, TOMS.

[30]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jian Gao,et al.  A coverage algorithm for multiple autonomous surface vehicles in flowing environments , 2016 .

[33]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.