A prototype of Enhanced Synthetic Vision System using short-wave infrared

ESVS (Enhanced Synthetic Vision System) carries out multisource data fusion during flight. It comprehensively processed and integrated FLIR, MMW, digital map and navigation data to output conformal and equivalent out-of-cabin view. ESVS assists pilots in seeing through complex atmosphere including haze, rain, snow, etc. With the help of ESVS, pilots can quickly identify runway, terrain and obstacle information in low visibility conditions, and achieve safety approaching and landing. We developed a prototype low cost ESVS based on short-wave infrared image sensor. Due to the atmospheric window of electromagnetic wave, the small dust particles and little droplets in the air are nearly transparent for the short-wave infrared (900nm-1700nm wavelength). Thus, it is very suitable for sensing surrounding environment in haze weather. Our ESVS focuses on seeing through haze weather condition during approaching of civil transport aircraft. In proposed ESVS, the forward-looking video sensor suite comprises a low-cost short-wave infrared camera and a visible color CCD camera. The CCD camera is mainly used to monitor the infrared data and evaluate the result of seeing through performance. With the guidance of navigation data, we made the multisource data fusion of short-wave infrared and synthetic vision to produce conform, neat ESVS visual display for pilots. In addition, we had offered real-time digital image enhancement, target recognition and Highway-In-The-Sky (HITS). This proposed ESVS will greatly improve the pilot vision in haze weather and has the potential to be equipped on the aircraft.

[1]  Gang He,et al.  Design and Flight Test of a Primary Flight Display Combined Vision System , 2011 .

[2]  R. D. Hayes,et al.  Synthetic Vision-a view in the fog , 1993 .

[3]  E. Theunissen,et al.  Synthetic vision - From simulation to flight testing , 2001 .

[4]  W. R. Jones,et al.  The NASA approach to realize a sensor enhanced-synthetic vision system (SE-SVS) [aircraft displays] , 2002, Proceedings. The 21st Digital Avionics Systems Conference.

[5]  Timothy J. Etherington,et al.  Synthetic Vision: Application Areas, Rationale, Implementation, and Results , 2009 .

[6]  Albert J. Ahumada,et al.  Engineering a visual system for seeing through fog , 1992 .

[7]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Joan Solà,et al.  Quaternion kinematics for the error-state Kalman filter , 2015, ArXiv.

[9]  G.J.M. Koeners,et al.  Integration of imaging sensor data into a synthetic vision display , 2004, The 23rd Digital Avionics Systems Conference (IEEE Cat. No.04CH37576).

[10]  R E Hobbs,et al.  Pictorial Format Display Evaluation , 1984 .

[11]  Bernd Korn,et al.  Combining Enhanced and Synthetic Vision for Autonomous All-Weather Approach and Landing , 2009 .

[12]  Randall E. Bailey,et al.  Commercial Flight Crew Decision Making During Low-Visibility Approach Operations Using Fused Synthetic and Enhanced Vision Systems , 2009 .

[13]  Lei Zhang,et al.  Parallel Image Detail Enhancement For Real-time Applications , 2015 .

[14]  David C. Foyle,et al.  Enhanced/Synthetic Vision Systems: Human Factors Research and Implications for Future Systems , 1992 .

[15]  Albert J. Ahumada,et al.  Sensor fusion for synthetic vision , 1991 .

[16]  W. W. Patterson.  Multiple function sensors for enhanced vision application , 1992, [1992] Proceedings IEEE/AIAA 11th Digital Avionics Systems Conference.