Illumination-invariant image matching for autonomous UAV localisation based on optical sensing

Abstract This paper presents an UAV (Unmanned Aerial Vehicle) localisation algorithm for its autonomous navigation based on matching between on-board UAV image sequences to a pre-installed reference satellite image. As the UAV images and the reference image are not necessarily taken under the same illumination condition, illumination-invariant image matching is essential. Based on the investigation of illumination-invariant property of Phase Correlation (PC) via mathematical derivation and experiments, we propose a PC based fast and robust illumination-invariant localisation algorithm for UAV navigation. The algorithm accurately determines the current UAV position as well as the next UAV position even the illumination condition of UAV on-board images is different from the reference satellite image. A Dirac delta function based registration quality assessment together with a risk alarming criterion is introduced to enable the UAV to perform self-correction in case the UAV deviates from the planned route. UAV navigation experiments using simulated terrain shading images and remote sensing images have demonstrated a robust high performance of the proposed PC based localisation algorithm under very different illumination conditions resulted from solar motion. The superiority of the algorithm, in comparison with two other widely used image matching algorithms, MI (Mutual Information) and NCC (Normalised Correlation Coefficient), is significant for its high matching accuracy and fast processing speed.

[1]  Jun Zhang,et al.  Novel Technique for Vision-Based UAV Navigation , 2011, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Jian Guo Liu,et al.  The Illumination Robustness of Phase Correlation for Image Alignment , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[3]  Larry H. Matthies,et al.  Two years of Visual Odometry on the Mars Exploration Rovers , 2007, J. Field Robotics.

[4]  Robert M. Haralick,et al.  Digital Step Edges from Zero Crossing of Second Directional Derivatives , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Michel Dhome,et al.  Towards geographical referencing of monocular SLAM reconstruction using 3D city models: Application to real-time accurate vision-based localization , 2009, CVPR.

[6]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[7]  Jung Ilkyun,et al.  Mobile Robot Navigation Using Difference of Wavelet SIFT , 2009, 2009 Second International Conference on Machine Vision.

[8]  Hugh Durrant-Whyte,et al.  Simultaneous localization and mapping (SLAM): part II , 2006 .

[9]  Jianguo Liu,et al.  Precise Subpixel Disparity Measurement From Very Narrow Baseline Stereo , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[10]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[11]  J. Thiran,et al.  Robust Phase-Correlation Based Registration of Airborne Videos Using Motion Estimation , 2013 .

[12]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[13]  B. N. Chatterji,et al.  An FFT-based technique for translation, rotation, and scale-invariant image registration , 1996, IEEE Trans. Image Process..

[14]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Miguel A. Olivares-Méndez,et al.  Computer Vision Onboard UAVs for Civilian Tasks , 2009, J. Intell. Robotic Syst..

[16]  Takeo Kanade,et al.  A visual odometer for autonomous helicopter flight , 1999, Robotics Auton. Syst..

[17]  Hassan Foroosh,et al.  Extension of phase correlation to subpixel registration , 2002, IEEE Trans. Image Process..

[18]  C. D. Kuglin,et al.  The phase correlation image alignment method , 1975 .

[19]  Alex Pentland,et al.  On the Imaging of Fractal Surfaces , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Greg Pisanich,et al.  Biologically inspired behavioral strategies for autonomous aerial explorers on Mars , 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652).

[21]  Gérard G. Medioni,et al.  Map-Enhanced UAV Image Sequence Registration and Synchronization of Multiple Image Sequences , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Yoram Bresler,et al.  On-line Vehicle Motion Estimation from Visual Terrain Information Part II: Ground Velocity and Position Estimation , 1986, IEEE Transactions on Aerospace and Electronic Systems.

[23]  Jan Wendel,et al.  An integrated GPS/MEMS-IMU navigation system for an autonomous helicopter , 2006 .

[24]  Eli Shechtman,et al.  Matching Local Self-Similarities across Images and Videos , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Jianguo Liu,et al.  Phase correlation pixel‐to‐pixel image co‐registration based on optical flow and median shift propagation , 2008 .

[26]  Zhongke Shi,et al.  Vision-based Runway Recognition for UAV Autonomous Landing , 2007 .

[27]  Gordon Wyeth,et al.  FAB-MAP + RatSLAM: Appearance-based SLAM for multiple times of day , 2010, 2010 IEEE International Conference on Robotics and Automation.

[28]  Michael Carsten Bosse,et al.  A VISION AUGMENTED NAVIGATION SYSTEM FOR AN AUTONOMOUS HELICOPTER , 1997 .

[29]  Mandyam V. Srinivasan,et al.  Bioinspired Engineering of Exploration Systems for NASA and DoD , 2002, Artificial Life.

[30]  Yoram Bresler,et al.  On-Line Vehicle Motion Estimation from Visual Terrain Information Part I: Recursive Image Registration , 1986, IEEE Transactions on Aerospace and Electronic Systems.

[31]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[32]  Lei Tian,et al.  Method for automatic georeferencing aerial remote sensing (RS) images from an unmanned aerial vehicle (UAV) platform , 2011 .

[33]  William Scott Hoge,et al.  A subspace identification extension to the phase correlation method [MRI application] , 2003, IEEE Transactions on Medical Imaging.

[34]  Xue Wan,et al.  PHASE CORRELATION BASED LOCAL ILLUMINATION-INVARIANT METHOD FOR MULTI-TEMPRO REMOTE SENSING IMAGE MATCHING , 2014 .

[35]  Jake K. Aggarwal,et al.  Matching Aerial Images to 3-D Terrain Maps , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Henrik Ohlsson,et al.  Geo-referencing for UAV navigation using environmental classification , 2010, 2010 IEEE International Conference on Robotics and Automation.