Monocular vision based obstacle avoidance trajectory planning for Unmanned Aerial Vehicle

Abstract In recent years, monocular vision has been considered as an effective Unmanned Aerial Vehicle (UAV) onboard obstacle perception solution with applications to Sense and Avoid (SAA). However, with the limitations of monocular optical measurement, monocular vision based obstacle localization ability is insufficient for collision avoidance. Therefore, an obstacle collision avoidance trajectory planning scheme is proposed in this paper with considerations of monocular optical measurement characteristic. Firstly, two obstacle localization modes are defined in this paper to assist with obstacle collision avoidance, namely relative range based Mode 1 and relative angle based Mode 2. Obstacle localization observability of Mode 2 is analyzed, and coordinate systems for obstacle localization are constructed based on the analysis results. Given coordinate systems, Orthogonal Iteration (OI) is further adopted for obstacle localization. Secondly, due to the lack of global knowledge caused by monocular vision based localization capability, a rolling horizon based safe trajectory planning method is presented. In each time segment, the trajectory is optimized with considerations of: 1) objective functions including minimum trajectory length, elapsed time and energy consumption; 2) constraints including Mode 1 and 2 based obstacle collision avoidance constrains and UAV flight constraints. Finally, simulation results indicate the proposed obstacle collision avoidance trajectory planning scheme enhances UAV safety level and can achieve favorable performance when compared with geometric based obstacle collision avoidance and collision avoidance with global knowledge.

[1]  Hyeongjun Park,et al.  Sub-optimal cooperative collision avoidance maneuvers of multiple active spacecraft via discrete-time generating functions , 2019, Aerospace Science and Technology.

[2]  Ning Zhang,et al.  A fast finite-time convergent guidance law with nonlinear disturbance observer for unmanned aerial vehicles collision avoidance , 2019, Aerospace Science and Technology.

[3]  Jason J. Ford,et al.  Vision-Based Estimation of Airborne Target Pseudobearing Rate using Hidden Markov Model Filters , 2013, IEEE Transactions on Aerospace and Electronic Systems.

[4]  Oisin Mac Aodha,et al.  Unsupervised Monocular Depth Estimation with Left-Right Consistency , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Michael J. Tribou,et al.  Observability of planar combined relative pose and target model estimation using monocular vision , 2010, Proceedings of the 2010 American Control Conference.

[6]  Narendra Ahuja,et al.  An analytically tractable potential field model of free space and its application in obstacle avoidance , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Lydia E. Kavraki,et al.  Randomized preprocessing of configuration for fast path planning , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[8]  Seiichi Shin,et al.  Self-organization of Decentralized Swarm Agents Based on Modified Particle Swarm Algorithm , 2006, J. Intell. Robotic Syst..

[9]  Zhaoyang Wang,et al.  Spatial and Temporal Context Information Fusion Based Flying Objects Detection for Autonomous Sense and Avoid , 2018, 2018 International Conference on Unmanned Aircraft Systems (ICUAS).

[10]  Shiqi Li,et al.  A Robust O(n) Solution to the Perspective-n-Point Problem , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Ngoc Hung Nguyen,et al.  Improved Pseudolinear Kalman Filter Algorithms for Bearings-Only Target Tracking , 2017, IEEE Transactions on Signal Processing.

[12]  Yunfeng Cao,et al.  An intruder detection algorithm for vision based sense and avoid system , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).

[13]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[14]  Brendan Harvey,et al.  Acoustic Detection of a Fixed-Wing UAV , 2018 .

[15]  Jihong Zhu,et al.  Topology Optimization in Aircraft and Aerospace Structures Design , 2016 .

[16]  Quan Pan,et al.  Feature article: Vision-based UAV collision avoidance with 2D dynamic safety envelope , 2016, IEEE Aerospace and Electronic Systems Magazine.

[17]  Srikanth Saripalli,et al.  Monocular Vision based Collaborative Localization for Micro Aerial Vehicle Swarms , 2018, 2018 International Conference on Unmanned Aircraft Systems (ICUAS).

[18]  Mingqing Xiao,et al.  Generic model of aircraft susceptibility to radar under conditions of electronic counter measures , 2016, IEEE Aerospace and Electronic Systems Magazine.

[19]  Aaron Mcfadyen,et al.  A Survey of autonomous vision-based See and Avoid for Unmanned Aircraft Systems , 2016 .

[20]  Rob Fergus,et al.  Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-scale Convolutional Architecture , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[22]  Dario Izzo,et al.  Persistent self-supervised learning: From stereo to monocular vision for obstacle avoidance , 2016, ArXiv.

[23]  Yuejin Zhao,et al.  Monocular vision avoidance method based on fully convolutional networks , 2018, SPIE/COS Photonics Asia.

[24]  Brian R. Geiger,et al.  Optimal Path Planning of UAVs Using Direct Collocation with Nonlinear Programming , 2006 .

[25]  Hok K. Ng,et al.  Aircraft Trajectory Optimization and Contrails Avoidance in the Presence of Winds , 2011 .

[26]  Lydia E. Kavraki,et al.  Probabilistic roadmaps for path planning in high-dimensional configuration spaces , 1996, IEEE Trans. Robotics Autom..

[27]  Gregory D. Hager,et al.  Fast and Globally Convergent Pose Estimation from Video Images , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Haibo Li,et al.  Candidate regions extraction of intruder airplane under complex background for vision-based sense and avoid system , 2017 .

[29]  Youmin Zhang,et al.  Sense and avoid technologies with applications to unmanned aircraft systems: Review and prospects , 2015 .

[30]  David P. Thipphavong,et al.  Analysis of Interactions Between Urban Air Mobility (UAM) Operations and Conventional Traffic in Urban Areas: Traffic Alert and Collision Avoidance System (TCAS) Study for UAM Operations , 2019 .

[31]  Yan Wan,et al.  Spatiotemporal scenario data-driven decision for the path planning of multiple UASs , 2019, SCOPE@CPSIoTWeek.

[32]  Yihong Wu,et al.  Image-based camera localization: an overview , 2016, Visual Computing for Industry, Biomedicine, and Art.

[33]  Jiwen Zheng.  C-Bézier curves and surfaces , 1999 .

[34]  I. Bar-Itzhack,et al.  Observability analysis of piece-wise constant systems with application to inertial navigation , 1990, 29th IEEE Conference on Decision and Control.

[35]  Peter Feil,et al.  Modular Demonstrator Test Bed for Airborne Sense & Avoid Radar , 2018, 2018 19th International Radar Symposium (IRS).

[36]  Guosheng Lin,et al.  Deep convolutional neural fields for depth estimation from a single image , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Stefan K. Gehrig,et al.  A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching , 2009, ICVS.

[38]  Nicu Sebe,et al.  Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Kimon P. Valavanis,et al.  Evolutionary algorithm based offline/online path planner for UAV navigation , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[40]  Masahiro Tomono,et al.  3-D Localization and Mapping Using a Single Camera Based on Structure-from-Motion with Automatic Baseline Selection , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[41]  Kim-Kwang Raymond Choo,et al.  Efficient Hierarchical Identity-Based Signature With Batch Verification for Automatic Dependent Surveillance-Broadcast System , 2017, IEEE Transactions on Information Forensics and Security.

[42]  Fuchun Sun,et al.  Recursive depth parametrization of monocular visual navigation: Observability analysis and performance evaluation , 2014, Inf. Sci..

[43]  B. H. Carson.  Fuel Efficiency of Small Aircraft , 1982 .

[44]  Don Ray Murray,et al.  Stereo vision based mapping and navigation for mobile robots , 1997, Proceedings of International Conference on Robotics and Automation.