Visual Measurement in Simulation Environment for Vision-Based UAV Autonomous Aerial Refueling

Autonomous aerial refueling (AAR) is an important capability of unmanned aerial vehicles (UAVs) for further applications in both military and civilian domains. This paper focuses on the description and comparison of visual measurement strategies in the simulation environment of AAR of UAVs. The visual measurement methods are employed for pose estimation between tanker and UAV receiver aircrafts. Within this effort, methods and techniques for marker detection, pose estimation, and visualization of aerial refueling in virtual reality have been designed and implemented. Series of comparative experiments indicate the preference of various visual measurement approaches in our work. Boom and stationkeeping controllers are designed to ensure system stability when wind turbulence is induced. Furthermore, the hardware-in-loop simulation platform is built to demonstrate the feasibility and effectiveness of the proposed scenarios in actual conditions. The visual attention mechanism is adopted in the preprocessing for the images of virtual scene to remove potential noises. The achieved results make excellent performances of our proposed scheme during the docking maneuver in AAR.

[1]  Wright-Patterson Afb,et al.  Automated Aerial Refueling: Extending the Effectiveness of Unmanned Air Vehicles , 2005 .

[2]  Francis K. H. Quek,et al.  Surface parameterization in volumetric images for curvature-based feature classification , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[3]  Chih-Lyang Hwang,et al.  Comparisons Between Two Visual Navigation Strategies for Kicking to Virtual Target Point of Humanoid Robots , 2013, IEEE Transactions on Instrumentation and Measurement.

[4]  Li Guo Jin,et al.  Camera calibration for monocular vision system based on Harris corner extraction and neural network , 2011, 2011 International Conference on Consumer Electronics, Communications and Networks (CECNet).

[5]  Binoy Pinto,et al.  Speeded Up Robust Features , 2011 .

[6]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[7]  Haibin Duan,et al.  Receding horizon control for multi-UAVs close formation control based on differential evolution , 2010, Science China Information Sciences.

[8]  Xiaohua Wang,et al.  Predator-Prey Biogeography-Based Optimization for Bio-inspired Visual Attention , 2013, Int. J. Comput. Intell. Syst..

[9]  M. L. Fravolini,et al.  Simulation Environment for Machine Vision Based Aerial Refueling for UAVs , 2009, IEEE Transactions on Aerospace and Electronic Systems.

[10]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[11]  Jonathan P. How,et al.  Operator Object Function Guidance for a Real-Time Unmanned Vehicle Scheduling Algorithm , 2012, J. Aerosp. Comput. Inf. Commun..

[12]  S. Q. Liu,et al.  Non-linear dual-mode receding horizon control for multiple unmanned air vehicles formation flight based on chaotic particle swarm optimisation , 2010 .

[13]  W. Eric L. Grimson,et al.  A shape-based approach to the segmentation of medical imagery using level sets , 2003, IEEE Transactions on Medical Imaging.

[14]  Monish D. Tandale,et al.  Trajectory Tracking Controller for Vision-Based Probe and Drogue Autonomous Aerial Refueling , 2005 .

[15]  Jiancheng Fang,et al.  Predictive Iterated Kalman Filter for INS/GPS Integration and Its Application to SAR Motion Compensation , 2010, IEEE Transactions on Instrumentation and Measurement.

[16]  Joe Nalepka,et al.  Automated Aerial Refueling: Extending the Effectiveness of UAVs , 2005 .

[17]  De Xu,et al.  Mixed Visual Control Method for Robots With Self-Calibrated Stereo Rig , 2010, IEEE Transactions on Instrumentation and Measurement.

[18]  Pierre Payeur,et al.  Application of Segmented 2-D Probabilistic Occupancy Maps for Robot Sensing and Navigation , 2008, IEEE Transactions on Instrumentation and Measurement.

[19]  Shahriar Keshmiri,et al.  The Meridian UAV Flight Performance Analysis Using Analytical and Experimental Data , 2009 .

[20]  James W. Kamman,et al.  Active control of aerial refueling hose-drogue systems , 2010 .

[21]  Gregory D. Hager,et al.  Fast and Globally Convergent Pose Estimation from Video Images , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Mario G. Perhinschi,et al.  Machine Vision/GPS Integration Using EKF for the UAV Aerial Refueling Problem , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[23]  Mark J. Monda,et al.  Boom and Receptacle Autonomous Air Refueling Using Visual Snake Optical Sensor , 2007 .

[24]  Jiwen Lu,et al.  Ordinary Preserving Manifold Analysis for Human Age and Head Pose Estimation , 2013, IEEE Transactions on Human-Machine Systems.

[25]  John Valasek,et al.  Vision based controller for autonomous aerial refueling , 2002, Proceedings of the International Conference on Control Applications.

[26]  Ya Tian,et al.  Accurate Human Navigation Using Wearable Monocular Visual and Inertial Sensors , 2014, IEEE Transactions on Instrumentation and Measurement.

[27]  Monish D. Tandale,et al.  Vision-Based Sensor and Navigation System for Autonomous Air Refueling , 2005 .

[28]  Baisravan Homchaudhuri,et al.  Cooperative Control of Multiple Uninhabited Aerial Vehicles for Monitoring and Fighting Wildfires , 2011, J. Aerosp. Comput. Inf. Commun..

[29]  Timothy W. McLain,et al.  Cooperative forest fire surveillance using a team of small unmanned air vehicles , 2006, Int. J. Syst. Sci..

[30]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[31]  Sivasankar Ramarasu.  Vision based navigation for an unmanned aerial vehicle , 2007 .

[32]  Atilla Dogan,et al.  Estimation of Receiver Aircraft States and Wind Vectors in Aerial Refueling , 2012 .

[33]  Zhuoning Dong,et al.  Multiple UCAVs cooperative air combat simulation platform based on PSO, ACO, and game theory , 2013, IEEE Aerospace and Electronic Systems Magazine.

[34]  J.K. Hedrick,et al.  Border patrol and surveillance missions using multiple unmanned air vehicles , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[35]  M. L. Fravolini,et al.  A comparison of Pose Estimation algorithms for Machine Vision based Aerial Refueling for UAVs , 2006, 2006 14th Mediterranean Conference on Control and Automation.

[36]  John Valasek,et al.  Vision Based Sensor and Navigation System for Autonomous Aerial Refueling , 2002 .

[37]  HaiBin Duan,et al.  Path planning of unmanned aerial vehicle based on improved gravitational search algorithm , 2012 .

[38]  Naser El-Sheimy,et al.  The Utilization of Artificial Neural Networks for Multisensor System Integration in Navigation and Positioning Instruments , 2006, IEEE Transactions on Instrumentation and Measurement.

[39]  Yuhui Shi,et al.  ?Hybrid Particle Swarm Optimization and Genetic Algorithm for Multi-UAV Formation Reconfiguration , 2013, IEEE Computational Intelligence Magazine.

[40]  Xiaohua Wang,et al.  Small and Dim Target Detection via Lateral Inhibition Filtering and Artificial Bee Colony Based Selective Visual Attention , 2013, PloS one.