Towards Automated Aerial Refueling: Real Time Position Estimation with Stereo Vision

Aerial refueling is essential to the United States Air Force (USAF) core mission of rapid global mobility. However, in-flight refueling is not available to remotely piloted aircraft (RPA) or unmanned aerial systems (UAS). As reliance on drones for intelligence, surveillance, and reconnaissance (ISR) and other USAF core missions grows, the ability to automate aerial refueling for such systems becomes increasingly critical. New refueling platforms include sensors that could be used to estimate the relative position of an approaching aircraft. Relative position estimation is a key component to solving the automated aerial refueling (AAR) problem. Analysis of data from a one-seventh scale, real world refueling scenario demonstrates that the relative position of an approaching aircraft can be estimated at rates between 10 Hz and 30 Hz using stereo vision. Linear regression models on position estimate accuracies predict results reported by other research in the simulation domain, suggesting that real world accuracies are comparable to simulation domain accuracies reported by others. Further, by seeding the position estimation algorithm with previous position estimates, subsequent errors in position estimation are reduced.

[1]  Kurt Konolige,et al.  Projected texture stereo , 2010, 2010 IEEE International Conference on Robotics and Automation.

[2]  S. Birchfiled.  A Pixel Dissimilarity Measure That Is Insensitive to Image Sampling , 1998 .

[3]  Mario G. Perhinschi,et al.  Machine Vision/GPS Integration Using EKF for the UAV Aerial Refueling Problem , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[4]  Zhengyou Zhang,et al.  Iterative point matching for registration of free-form curves and surfaces , 1994, International Journal of Computer Vision.

[5]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[6]  Glenn Bever,et al.  COMPARISON OF RELATIVE NAVIGATION SOLUTIONS APPLIED BETWEEN TWO AIRCRAFT , 2002 .

[7]  Mario G. Perhinschi,et al.  Machine Vision Algorithms for Autonomous Aerial Refueling for UAVs Using the USAF Refueling Boom Method , 2010 .

[8]  Marcello R. Napolitano,et al.  Comparison of point matching algorithms for the UAV aerial refueling problem , 2010, Machine Vision and Applications.

[9]  Joseph A Curro.  Automated Aerial Refueling Position Estimation Using a Scanning LiDAR , 2012 .

[10]  Michael Veth,et al.  Fusion of Imaging and Inertial Sensors for Navigation , 2006 .

[11]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[13]  Justin Tharp.  On the Integration of Medium Wave Infrared Cameras for Vision-Based Navigation , 2015 .

[14]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[15]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[16]  Steven M Ross.  Formation Flight Control for Aerial Refueling , 2012 .

[17]  Marcello R. Napolitano,et al.  GPS / MV based Aerial Refueling for UAVs , 2008 .

[18]  Eun-Jung Song,et al.  AN INSTRUMENTATION SYSTEM APPLIED TO FORMATION FLIGHT , 2002 .

[19]  Narendra Ahuja,et al.  A constant-space belief propagation algorithm for stereo matching , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Joseph L. Tchon,et al.  3D display considerations for rugged airborne environments , 2015, Defense + Security Symposium.

[21]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[22]  Richard I. Hartley,et al.  Theory and Practice of Projective Rectification , 1999, International Journal of Computer Vision.

[23]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[24]  Kyle P Werner,et al.  Precision Relative Positioning for Automated Aerial Refueling from a Stereo Imaging System , 2015 .

[25]  B. Cyganek.  An Introduction to 3D Computer Vision Techniques and Algorithms , 2009 .

[26]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[27]  Paul J. Besl,et al.  Method for registration of 3-D shapes , 1992, Other Conferences.

[28]  Sean M. Calhoun,et al.  Flight Test Evaluation of Image Rendering Navigation for Close-Formation Flight , 2012 .

[29]  Marcello R. Napolitano,et al.  Evaluation of Machine Vision Algorithms for Autonomous Aerial Refueling for Unmanned Aerial Vehicles , 2007, J. Aerosp. Comput. Inf. Commun..