Drogue pose estimation for unmanned aerial vehicle autonomous aerial refueling system based on infrared vision sensor

Abstract. Autonomous aerial refueling is a significant technology that can significantly extend the endurance of unmanned aerial vehicles. A reliable method that can accurately estimate the position and attitude of the probe relative to the drogue is the key to such a capability. A drogue pose estimation method based on infrared vision sensor is introduced with the general goal of yielding an accurate and reliable drogue state estimate. First, by employing direct least squares ellipse fitting and convex hull in OpenCV, a feature point matching and interference point elimination method is proposed. In addition, considering the conditions that some infrared LEDs are damaged or occluded, a missing point estimation method based on perspective transformation and affine transformation is designed. Finally, an accurate and robust pose estimation algorithm improved by the runner-root algorithm is proposed. The feasibility of the designed visual measurement system is demonstrated by flight test, and the results indicate that our proposed method enables precise and reliable pose estimation of the probe relative to the drogue, even in some poor conditions.

[1]  Haibin Duan,et al.  Visual Measurement in Simulation Environment for Vision-Based UAV Autonomous Aerial Refueling , 2015, IEEE Transactions on Instrumentation and Measurement.

[2]  Martin Buss,et al.  Visual tracking and control of a quadcopter using a stereo camera system and inertial sensors , 2009, 2009 International Conference on Mechatronics and Automation.

[3]  Monish D. Tandale,et al.  Vision-Based Sensor and Navigation System for Autonomous Air Refueling , 2005 .

[4]  Salah Sukkarieh,et al.  Experimental validation of a drogue estimation algorithm for autonomous aerial refueling , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Mario Innocenti,et al.  Vision Algorithms for Formation Flight and Aerial Refueling with Optimal Marker Labeling , 2005 .

[6]  J.A.A. Engelbrecht,et al.  Receptacle normal position control for automated aerial refueling , 2013 .

[7]  Farshad Merrikh-Bayat,et al.  The runner-root algorithm , 2015 .

[8]  M. L. Fravolini,et al.  Simulation Environment for Machine Vision Based Aerial Refueling for UAVs , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[9]  Xiaoping Du,et al.  Scale-unambiguous relative pose estimation of space uncooperative targets based on the fusion of three-dimensional time-of-flight camera and monocular camera , 2015 .

[10]  Shiqi Li,et al.  A Robust O(n) Solution to the Perspective-n-Point Problem , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Pascual Campoy Cervera,et al.  A vision-based strategy for autonomous aerial refueling tasks , 2013, Robotics Auton. Syst..

[12]  Mark J. Monda,et al.  Boom and Receptacle Autonomous Air Refueling Using Visual Snake Optical Sensor , 2007 .

[13]  Ning Xian,et al.  Biological object recognition approach using space variant resolution and pigeon-inspired optimization for UAV , 2017 .

[14]  D. Scott,et al.  Vision-based sensing for autonomous in-flight refueling , 2007, SPIE Defense + Commercial Sensing.

[15]  Xiangyi Sun,et al.  The relative pose estimation of aircraft based on contour model , 2017, Other Conferences.

[16]  Salah Sukkarieh,et al.  Drogue motion estimation using air-to-air observations , 2014 .

[17]  Marc Schwarzbach,et al.  An infrared vision system for UAV close formation flight , 2015 .

[18]  Yong Chen,et al.  Real-time drogue recognition and 3D locating for UAV autonomous aerial refueling based on monocular machine vision , 2015 .

[19]  Gregory D. Hager,et al.  Fast and Globally Convergent Pose Estimation from Video Images , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Delin Luo,et al.  A guidance law for UAV autonomous aerial refueling based on the iterative computation method , 2014 .

[21]  Pei Li,et al.  A binocular vision-based UAVs autonomous aerial refueling platform , 2016, Science China Information Sciences.

[22]  Jing Zhang,et al.  Improved iterative pose estimation algorithm using three-dimensional feature points , 2007 .

[23]  De Xu,et al.  Detection and Tracking Strategies for Autonomous Aerial Refuelling Tasks Based on Monocular Vision , 2014 .

[24]  Steve E Bullock,et al.  Advances in air to air refuelling , 2014 .

[25]  Bodo Rosenhahn,et al.  Pose Estimation of 3D Free-Form Contours , 2005, International Journal of Computer Vision.