Robust visual-inertial odometry with point and line features for blade inspection UAV

Purpose This paper aims to enable the unmanned aerial vehicles to inspect the surface condition of wind turbine in close range when the global positioning system signal is not reliable, and further improve its intelligence. So a visual-inertial odometry with point and line features is developed. Design/methodology/approach Visual front-end combining point and line features, as well as its purification strategies, are first presented to improve the robustness of feature tracking in low-textured scene and rapidity of segment detector. Additionally, the inertial measurement is integrated between keyframes as constrain to reduce tracking error existed in visual-only system. Second, the graph-based visual-inertial back-end is constructed. To parameterize line features effectively, the infinite line representation not sensitive to outdoor light is employed, in which Plücker and Cayley are selected for line re-projection and nonlinear optimization. Furthermore, Jacobians of the line re-projection errors are analytically derived for better accuracy. Findings Experiments are performed in various scenes of the wind farm. The results demonstrate that the tight-coupled visual-inertial odometry with point and line features is more precise on all the samples than conventional algorithms in complex wind farm environments. Additionally, the constructed line feature map can be used in the following research for autonomous navigation. Originality/value The proposed visual-inertial odometry works robustly in strong electromagnetic interference, low-textured and illumination-change wind farm.

[1]  Reinhard Koch,et al.  Structure and motion from line correspondences: Representation, projection, initialization and sparse bundle adjustment , 2014, J. Vis. Commun. Image Represent..

[2]  Frank Dellaert,et al.  On-Manifold Preintegration for Real-Time Visual--Inertial Odometry , 2015, IEEE Transactions on Robotics.

[3]  Shaojie Shen,et al.  VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator , 2017, IEEE Transactions on Robotics.

[4]  Wendy Flores-Fuentes,et al.  Exact laser beam positioning for measurement of vegetation vitality , 2017, Ind. Robot.

[5]  Jiamao Li,et al.  Stereo Visual-Inertial SLAM With Points and Lines , 2018, IEEE Access.

[6]  Rafael Grompone von Gioi,et al.  LSD: A Fast Line Segment Detector with a False Detection Control , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Dirk Abel,et al.  Multicopter unmanned aerial vehicle for automated inspection of wind turbines , 2016, 2016 24th Mediterranean Conference on Control and Automation (MED).

[8]  Adrien Bartoli,et al.  Structure-from-motion using lines: Representation, triangulation, and bundle adjustment , 2005, Comput. Vis. Image Underst..

[9]  Yong Liu,et al.  Robust visual SLAM with point and line features , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[10]  Andrew Calway,et al.  Improving drone localisation around wind turbines using monocular model-based tracking , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[11]  Salah Sukkarieh,et al.  Visual-Inertial-Aided Navigation for High-Dynamic Motion in Built Environments Without Initial Conditions , 2012, IEEE Transactions on Robotics.

[12]  Tor A. Johansen,et al.  Autonomous visual navigation of Unmanned Aerial Vehicle for wind turbine inspection , 2015, 2015 International Conference on Unmanned Aircraft Systems (ICUAS).

[13]  John J. Leonard,et al.  Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age , 2016, IEEE Transactions on Robotics.

[14]  Reinhard Koch,et al.  An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency , 2013, J. Vis. Commun. Image Represent..

[15]  Juan D. Tardós,et al.  Visual-Inertial Monocular SLAM With Map Reuse , 2016, IEEE Robotics and Automation Letters.

[16]  Teresa A. Vidal-Calleja,et al.  Impact of Landmark Parametrization on Monocular EKF-SLAM with Points and Lines , 2011, International Journal of Computer Vision.

[17]  Javier Gonzalez-Jimenez,et al.  PL-SLAM: A Stereo SLAM System Through the Combination of Points and Line Segments , 2017, IEEE Transactions on Robotics.

[18]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[19]  John J. Leonard,et al.  Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age , 2016, IEEE Transactions on Robotics.

[20]  Zijun Zhang,et al.  Automatic Detection of Wind Turbine Blade Surface Cracks Based on UAV-Taken Images , 2017, IEEE Transactions on Industrial Electronics.

[21]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[22]  Cuneyt Akinlar,et al.  Edge Drawing: A combined real-time edge and segment detector , 2012, J. Vis. Commun. Image Represent..