Deep learning enabled localization for UAV autolanding

Abstract This article concentrates on ground vision guided autonomous landing of a fixed-wing Unmanned Aerial Vehicle (UAV) within Global Navigation Satellite System (GNSS) denied environments. Cascaded deep learning models are developed and employed into image detection and its accuracy promoting for UAV autolanding, respectively. Firstly, we design a target bounding box detection network BboxLocate-Net to extract its image coordinate of the flying object. Secondly, the detected coordinate is fused into spatial localization with an extended Kalman filter estimator. Thirdly, a point regression network PointRefine-Net is developed for promoting detection accuracy once the flying vehicle’s motion continuity is checked unacceptable. The proposed approach definitely accomplishes the closed-loop mutual inspection of spatial positioning and image detection, and automatically improves the inaccurate coordinates within a certain range. Experimental results demonstrate and verify that our method outperforms the previous works in terms of accuracy, robustness and real-time criterions. Specifically, the newly developed BboxLocate-Net attaches over 500 fps, almost five times the published state-of-the-art in this field, with comparable localization accuracy.

[1]  Dong Liu,et al.  Deep High-Resolution Representation Learning for Human Pose Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Tianjiang Hu,et al.  Chan-Vese model based binocular visual object extraction for UAV autonomous take-off and landing , 2015, 2015 5th International Conference on Information Science and Technology (ICIST).

[3]  Tianjiang Hu,et al.  ROS-based ground stereo vision detection: implementation and experiments , 2016, Robotics and biomimetics.

[4]  Xiang Zhou,et al.  Airborne Vision-Based Navigation Method for UAV Accuracy Landing Using Infrared Lamps , 2013, J. Intell. Robotic Syst..

[5]  Ridong Zhu,et al.  Real-time object tracking via least squares transformation in spatial and Fourier domains for unmanned aerial vehicles , 2019 .

[6]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[7]  Tianjiang Hu,et al.  Ground Stereo Vision-Based Navigation for Autonomous Take-off and Landing of UAVs: A Chan-Vese Model Approach , 2016 .

[8]  Yu Zhang,et al.  A ground-based optical system for autonomous landing of a fixed wing UAV , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[10]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[11]  Alexandre Bernardino,et al.  Autoland project: Fixed-wing UAV Landing on a Fast Patrol Boat using Computer Vision , 2019, OCEANS 2019 MTS/IEEE SEATTLE.

[12]  Tianjiang Hu,et al.  Enabling ∼100fps detection on a landing unmanned aircraft for its on-ground vision-based recovery , 2017, 2017 18th International Conference on Advanced Robotics (ICAR).

[13]  Daibing Zhang,et al.  Ground-based visual guidance in autonomous UAV landing , 2013, Other Conferences.

[14]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[15]  Tianjiang Hu,et al.  Stereo Vision Guiding for the Autonomous Landing of Fixed-Wing UAVs: A Saliency-Inspired Approach , 2016 .

[16]  Tao Zhang,et al.  A vision-based navigation approach with multiple radial shape marks for indoor aircraft locating , 2014 .

[17]  William K. Durfee,et al.  IEEE/RSJ/GI International Conference on Intelligent Robots and Systems , 1994 .

[18]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Jianwei Zhang,et al.  Autonomous landing of an UAV with a ground-based actuated infrared stereo vision system , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Xiaolin Tang,et al.  A novel simplified model for torsional vibration analysis of a series-parallel hybrid electric vehicle , 2017 .

[21]  Kong Weiwei,et al.  Autonomous control of running takeoff and landing for a fixed-wing unmanned aerial vehicle , 2012, 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV).

[22]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[23]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[24]  Р Ю Чуйков,et al.  Обнаружение транспортных средств на изображениях загородных шоссе на основе метода Single shot multibox Detector , 2017 .

[25]  François Chaumette,et al.  Visual servoing of an airplane for auto-landing , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Kenzo Nonami,et al.  Autonomous Hovering and Landing of a Quad-rotor Micro Aerial Vehicle by Means of on Ground Stereo Vision System , 2010 .

[27]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[28]  Xiaohu Zhang,et al.  Multi-camera networks for motion parameter estimation of an aircraft , 2017 .

[29]  Vijay Kumar,et al.  Opportunities and challenges with autonomous micro aerial vehicles , 2012, Int. J. Robotics Res..

[30]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Yanning Zhang,et al.  A Ground-Based Near Infrared Camera Array System for UAV Auto-Landing in GPS-Denied Environment , 2016, Sensors.