Augmented Reality in a Remote Tower Environment Based on VS/IR Fusion and Optical Tracking

Over the past years, several augmented reality features have been developed to make Remote Tower Operations more cost-efficient and user-friendly. In the context of a national research project (The paper reports results gained in the project “INVIDEON” (FKZ 20 V1505A) sponsored by the Luftfahrtforschungsprogramm (LuFo) of the Federal Ministry of Transport and Digital Infrastructure Germany.), augmented reality based on visual spectrum (VS) and infrared (IR) fusion and as well as on optical tracking is a study objective. Having both VS and IR information available at any time is expected to enable more efficient air traffic control, even at restricted visibility conditions. Integrating VS and IR in one video panorama should also decrease head-down times and therefore increase situation awareness and reduce workload. The integration of two different sensors will be realized by overlaying VS/IR combined with adapted input devices and optical tracking methods. Developing a good concept for the integration of VS/IR and testing it in an exploratory manner can only be achieved with the help of system experts and rapid prototyping methods in simulation environments. During three workshops, human factors specialists, project partners and seven air traffic controllers worked out a prototype that was gradually improved over time and helped to generate a first concept. Firstly, this paper addresses the challenges of VS/IR fusion, manual PTZ following (as a precursor for optical tracking) and adapted input devices. Secondly, it presents the construction process of a prototype in an explorative manner, based on a user-centered approach and implemented in a simulation environment. Finally, it summarizes and presents the results from the workshops and throughout the construction process.

[1]  J. French,et al.  Overcoming Resistance to Change , 1948 .

[2]  C. Mohlenbrink,et al.  Remote airport traffic control center with augmented vision video panorama , 2009, 2009 IEEE/AIAA 28th Digital Avionics Systems Conference.

[3]  Bernard D. Adelstein,et al.  Augmented Reality in a Simulated Tower Environment: Effect of Field of View on Aircraft Detection , 2002 .

[4]  Anne Papenfuss,et al.  Head up only — A design concept to enable multiple remote tower operations , 2016, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC).

[5]  J. Tanaka,et al.  Color diagnosticity in object recognition , 1999, Perception & psychophysics.

[6]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[7]  Sebastian Schier,et al.  Visual scanning in an air traffic control tower – A simulation study , 2015 .

[8]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[9]  Norbert Fürstenau,et al.  Augmented Vision Videopanorama System for Remote Airport Tower Operation , 2008 .

[10]  R.D. Hudson,et al.  The military applications of remote sensing by infrared , 1975, Proceedings of the IEEE.

[11]  Ruzena Bajcsy,et al.  Texture gradient as a depth cue , 1976 .

[12]  Ronald Reisman,et al.  Design of Augmented Reality Tools for Air Traffic Control Towers , 2006 .

[13]  A. Rogalski.  Infrared detectors: an overview , 2002 .

[14]  Doris M Dehn,et al.  Assessing the Impact of Automation on the Air Traffic Controller: The SHAPE Questionnaires , 2008 .

[15]  Christina König,et al.  How to reduce workload--augmented reality to ease the work of air traffic controllers. , 2012, Work.

[16]  Greg Welch,et al.  Testing and evaluation of a wearable augmented reality system for natural outdoor environments , 2013, Defense, Security, and Sensing.

[17]  David Alleysson.  Le traitement du signal chromatique dans la rétine : Un modèle de base pour la perception humaine des couleurs. , 1999 .

[18]  Stephen R. Ellis.  Towards Determination of Visual Requirements for Augmented Reality Displays and Virtual Environments for the Airport Tower , 2006 .

[19]  Introduction and Overview 1.1 Introduction , 2001 .

[20]  Emmanuel Zenou,et al.  Active and thermal imaging performance under bad weather conditions , 2014 .

[21]  Norbert Fürstenau,et al.  On the use of transparent rear projection screens to reduce head-down time in the air-traffic control tower , 2004 .

[22]  Emmanuel Zenou,et al.  Experiments and models of active and thermal imaging under bad weather conditions , 2013, Optics/Photonics in Security and Defence.