Infrared-Inertial Navigation for Commercial Aircraft Precision Landing in Low Visibility and GPS-Denied Environments

This paper proposes a novel infrared-inertial navigation method for the precise landing of commercial aircraft in low visibility and Global Position System (GPS)-denied environments. Within a Square-root Unscented Kalman Filter (SR_UKF), inertial measurement unit (IMU) data, forward-looking infrared (FLIR) images and airport geo-information are integrated to estimate the position, velocity and attitude of the aircraft during landing. Homography between the synthetic image and the real image which implicates the camera pose deviations is created as vision measurement. To accurately extract real runway features, the current results of runway detection are used as the prior knowledge for the next frame detection. To avoid possible homography decomposition solutions, it is directly converted to a vector and fed to the SR_UKF. Moreover, the proposed navigation system is proven to be observable by nonlinear observability analysis. Last but not least, a general aircraft was elaborately equipped with vision and inertial sensors to collect flight data for algorithm verification. The experimental results have demonstrated that the proposed method could be used for the precise landing of commercial aircraft in low visibility and GPS-denied environments.

[1]  Gaurav S. Sukhatme,et al.  Visual-Inertial Sensor Fusion: Localization, Mapping and Sensor-to-Sensor Self-calibration , 2011, Int. J. Robotics Res..

[2]  Tianjiang Hu,et al.  Ground Stereo Vision-Based Navigation for Autonomous Take-off and Landing of UAVs: A Chan-Vese Model Approach , 2016 .

[3]  Xiang Zhou,et al.  Airborne Vision-Based Navigation Method for UAV Accuracy Landing Using Infrared Lamps , 2013, J. Intell. Robotic Syst..

[4]  Vijay Kumar,et al.  Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight , 2017, IEEE Robotics and Automation Letters.

[5]  Miti Ruchanurucks,et al.  Automatic Landing Assist System Using IMU+PnP for Robust Positioning of Fixed-Wing UAVs , 2018, J. Intell. Robotic Syst..

[6]  Yanming Fan,et al.  Vision algorithms for fixed-wing unmanned aerial vehicle landing system , 2017 .

[7]  Zhihui Lei,et al.  An Independently Carrier Landing Method Using Point and Line Features for Fixed-Wing UAVs , 2016, IGTA.

[8]  R. A. Gray,et al.  An integrated GPS/INS/baro and radar altimeter system for aircraft precision approach landings , 1995, Proceedings of the IEEE 1995 National Aerospace and Electronics Conference. NAECON 1995.

[9]  Jiyun Lee,et al.  Optimized GNSS Station Selection to Support Long-Term Monitoring of Ionospheric Anomalies for Aircraft Landing Systems , 2017, IEEE Transactions on Aerospace and Electronic Systems.

[10]  Tianjiang Hu,et al.  Stereo Vision Guiding for the Autonomous Landing of Fixed-Wing UAVs: A Saliency-Inspired Approach , 2016 .

[11]  Jean-Michel Pflimlin,et al.  Automatic landing on aircraft carrier by visual servoing , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Sreenatha G. Anavatti,et al.  Visual–Inertial Navigation Systems for Aerial Robotics: Sensor Fusion and Technology , 2017, IEEE Transactions on Automation Science and Engineering.

[13]  Franck Plestan,et al.  Nonlinear observers in vision system: Application to civil aircraft landing , 2015, 2015 European Control Conference (ECC).

[14]  Xin Li,et al.  Airborne vision-aided landing navigation system for fixed-wing UAV , 2014, 2014 12th International Conference on Signal Processing (ICSP).

[15]  S. Shankar Sastry,et al.  An Invitation to 3-D Vision: From Images to Geometric Models , 2003 .

[16]  S. K. Kashyap,et al.  Detection of Runway and Obstacles using Electro-optical and Infrared Sensors before Landing , 2014 .

[17]  Franck Plestan,et al.  Visual estimation of deviations for the civil aircraft landing , 2018 .

[18]  Rudolph van der Merwe,et al.  The square-root unscented Kalman filter for state and parameter-estimation , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[19]  Laurent Burlion,et al.  Toward vision based landing of a fixed-wing UAV on an unknown runway under some fov constraints , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[20]  Lei Zhang,et al.  Real-time Accurate Runway Detection based on Airborne Multi-sensors Fusion , 2017 .

[21]  Stefano Soatto,et al.  Visual-Inertial-Semantic Scene Representation for 3D Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  François Chaumette,et al.  Visual detection and 3D model-based tracking for landing on an aircraft carrier , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  Stephan Weiss,et al.  Vision based navigation for micro helicopters , 2012 .

[24]  Franck Plestan,et al.  New pose estimation scheme in perspective vision system during civil aircraft landing , 2015, SyRoCo.

[25]  Yan Huang,et al.  Infrared and Visible Image Fusion Based on Different Constraints in the Non-Subsampled Shearlet Transform Domain , 2018, Sensors.

[26]  Yang Xiao,et al.  Efficient Airport Detection Using Line Segment Detector and Fisher Vector Representation , 2016, IEEE Geoscience and Remote Sensing Letters.

[27]  Fei Gao,et al.  Real-time monocular dense mapping on aerial robots using visual-inertial fusion , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[29]  Yu Zhang,et al.  A ground-based optical system for autonomous landing of a fixed wing UAV , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Yanning Zhang,et al.  A Ground-Based Near Infrared Camera Array System for UAV Auto-Landing in GPS-Denied Environment , 2016, Sensors.

[31]  Miguel A. Olivares-Méndez,et al.  On-board and Ground Visual Pose Estimation Techniques for UAV Control , 2011, J. Intell. Robotic Syst..

[32]  Paulo Vinicius Koerich Borges,et al.  Practical Infrared Visual Odometry , 2016, IEEE Transactions on Intelligent Transportation Systems.

[33]  Jianwei Zhang,et al.  Autonomous landing of an UAV with a ground-based actuated infrared stereo vision system , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[34]  Konstantin Kondak,et al.  Landing of a fixed-wing UAV on a mobile ground vehicle , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[35]  Cuneyt Akinlar,et al.  EDLines: A real-time line segment detector with a false detection control , 2011, Pattern Recognit. Lett..

[36]  Robert E. Mahony,et al.  Approach maneuvers for autonomous landing using visual servo control , 2014, IEEE Transactions on Aerospace and Electronic Systems.

[37]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Flavio Fontana,et al.  Simultaneous State Initialization and Gyroscope Bias Calibration in Visual Inertial Aided Navigation , 2017, IEEE Robotics and Automation Letters.

[39]  P. B. Sujit,et al.  A survey of autonomous landing techniques for UAVs , 2014, 2014 International Conference on Unmanned Aircraft Systems (ICUAS).

[40]  Shaojie Shen,et al.  Monocular Visual–Inertial State Estimation With Online Initialization and Camera–IMU Extrinsic Calibration , 2017, IEEE Transactions on Automation Science and Engineering.

[41]  Hans-Ullrich Doehler,et al.  Autonomous infrared-based guidance system for approach and landing , 2004, SPIE Defense + Commercial Sensing.

[42]  Patrick Rives,et al.  Vision-Based Automatic Approach and Landing of Fixed-Wing Aircraft Using a Dense Visual Tracking , 2011 .

[43]  Juan D. Tardós,et al.  Visual-Inertial Monocular SLAM With Map Reuse , 2016, IEEE Robotics and Automation Letters.

[44]  Jia Ning,et al.  An Independently Carrier Landing Method Using Point and Line Features for Fixed-Wing UAVs , 2016 .

[45]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[46]  Jianwei Zhang,et al.  Localization Framework for Real-Time UAV Autonomous Landing: An On-Ground Deployed Visual Approach , 2017, Sensors.

[47]  E. Malis,et al.  Deeper understanding of the homography decomposition for vision-based control , 2007 .

[48]  A. Krener,et al.  Nonlinear controllability and observability , 1977 .

[49]  Wassim Khalaf,et al.  Novel adaptive UKF for tightly-coupled INS/GPS integration with experimental validation on an UAV , 2017 .

[50]  Kui Li,et al.  FLIR/INS/RA Integrated Landing Guidance for Landing on Aircraft Carrier , 2015 .

[51]  Changjiang Liu,et al.  Runway Extraction in Low Visibility Conditions Based on Sensor Fusion Method , 2014, IEEE Sensors Journal.

[52]  Jianwei Zhang,et al.  Vision-based autonomous landing system for unmanned aerial vehicle: A survey , 2014, 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems (MFI).