Effects of auditory and visual feedback on remote pilot manual flying performance

Abstract Remotely Piloted Aircraft Systems (RPAS) have facilitated new growth in civil aviation. Unlike manned aircraft, however, they are operated without auditory feedback and normally flown under two visual conditions: in direct visual-line-of-sight to the remote pilot (VLOS) and beyond VLOS with first-person-view imagery transmitted via onboard cameras (BVLOS). The present research examined the effectiveness of audiovisual cueing on remote pilot manual flying performance. Eighteen pilots (three female) completed six navigation and 12 spotting tasks. Their flying performance (horizontal accuracy, vertical accuracy and timeliness) was examined under three different visual display types (VLOS (Control), BVLOS-Monitor & BVLOS-Goggles), with and without real-time auditory feedback, and two wind component (no wind and wind) conditions. Horizontal deviation and timeliness improved in the BVLOS-Monitor condition navigation task, while auditory feedback produced nuanced examples of improved and degraded pilot performance. These results indicate how the specificity of the task, combined with different levels of audiovisual feedback influences remote pilot performance. These findings support the rationalisation for the provision of multimodal dynamic sensory cueing in future RPAS. Practitioner summary: Accuracy and timeliness of remote pilot manual flying performance was measured under a combination of audiovisual feedback in calm and wind shear conditions. The inclusion of real-time auditory feedback as an additional sensory cue is uncommon; this study demonstrated nuanced examples of improved and degraded manual flying performance. The provision of dynamic sensory cueing made available to remote pilots in future RPAS should be considered.

[1]  Sebastian Günther,et al.  Exploring Audio, Visual, and Tactile Cues for Synchronous Remote Assistance , 2018, PETRA.

[2]  G. Recanzone.  Auditory influences on visual temporal rate perception. , 2003, Journal of neurophysiology.

[3]  Anders Kjellberg,et al.  Poor Listening Conditions Impair Memory for Intelligible Lectures: Implications for Acoustic Classroom Standards , 2009 .

[4]  M Akamatsu,et al.  Please Scroll down for Article Ergonomics a Comparison of Tactile, Auditory, and Visual Feedback in a Pointing Task Using a Mouse-type Device , 2022 .

[5]  B. Bałaj,et al.  Selective Auditory Attention and Spatial Disorientation Cues Effect on Flight Performance. , 2018, Aerospace medicine and human performance.

[6]  Shahram Sarkani,et al.  Minimizing Human Factors Mishaps in Unmanned Aircraft Systems , 2013 .

[7]  Richard A Armstrong,et al.  When to use the Bonferroni correction , 2014, Ophthalmic & physiological optics : the journal of the British College of Ophthalmic Opticians.

[8]  Justin A Jaussi,et al.  Manned Versus Unmanned Aircraft Accidents, Including Causation and Rates , 2018 .

[9]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[10]  Brett R.C. Molesworth,et al.  The Noise Reduction Provided by Aviation Headsets , 2016 .

[11]  Lars Eriksson,et al.  Enhanced Perception and Performance by Multimodal Threat Cueing in Simulated Combat Vehicle , 2012, Hum. Factors.

[12]  Richard Kronland-Martinet,et al.  How does interior car noise alter driver's perception of motion? Multisensory integration in speed perception , 2012 .

[13]  Diana Löffler,et al.  The effect on recognition memory of noise cancelling headphones in a noisy environment with native and non-native speakers. , 2014, Noise & health.

[14]  Alexandra Neukum,et al.  The effect of urgency of take-over requests during highly automated driving under distraction conditions , 2014 .

[15]  N. Bolognini,et al.  Enhancement of visual perception by crossmodal visuo-auditory interaction , 2002, Experimental Brain Research.

[16]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[17]  Henry Been-Lirn Duh,et al.  Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment , 2002, Proceedings IEEE Virtual Reality 2002.

[18]  Alan Hobbs,et al.  Human Factors Guidelines for Unmanned Aircraft Systems , 2016 .

[19]  Brian P. Self,et al.  10. Spatial Disorientation in Uninhabited Aerial Vehicles , 2006 .

[20]  C. Spence,et al.  Assessing the role of stimulus probability on the Colavita visual dominance effect , 2007, Neuroscience Letters.

[21]  M. Posner,et al.  Visual dominance: an information-processing account of its origins and significance. , 1976, Psychological review.

[22]  Marion Burgess,et al.  The effects of noise on key workplace skills. , 2015, The Journal of the Acoustical Society of America.

[23]  Walter W. Wierwille,et al.  The Influence of Motion and Audio Cues on Driver Performance in an Automobile Simulator , 1975 .

[24]  Anjum Naweed,et al.  The influence of auditory feedback on speed choice, violations and comfort in a driving simulation game , 2011 .

[25]  Brett R.C. Molesworth,et al.  The use of noise cancelling headphones to improve concurrent task performance in a noisy environment , 2013 .

[26]  Kevin W Williams,et al.  A Summary of Unmanned Aircraft Accident/Incident Data: Human Factors Implications , 2004 .

[27]  F. Colavita.  Human sensory dominance , 1974 .

[28]  P A Hancock,et al.  Effects of jet engine noise and performance feedback on perceived workload in a monitoring task. , 1995, The International journal of aviation psychology.

[29]  G. Pfotzer,et al.  History of the use of balloons in scientific experiments , 1972 .

[30]  C. Spence.  Explaining the Colavita visual dominance effect. , 2009, Progress in brain research.

[31]  K J Rothman,et al.  No Adjustments Are Needed for Multiple Comparisons , 1990, Epidemiology.

[32]  Natasha Merat,et al.  A Driving Simulator Study to Examine the Role of Vehicle Acoustics on Drivers' Speed Perception , 2017 .

[33]  Thomas A. Stoffregen,et al.  The virtual reality head-mounted display Oculus Rift induces motion sickness and is sexist in its effects , 2017, Experimental Brain Research.

[34]  H. J. Arnold.  Introduction to the Practice of Statistics , 1990 .

[35]  Christopher B. Mayhorn,et al.  Multimodal Cueing: The Relative Benefits of the Auditory, Visual, and Tactile Channels in Complex Environments , 2012 .

[36]  C. Spence,et al.  Audiovisual asynchrony modulates the Colavita visual dominance effect , 2007, Brain Research.

[37]  M. Reiner,et al.  Sensory dominance in combinations of audio, visual and haptic stimuli , 2009, Experimental Brain Research.