Latency requirements for head-worn display S/EVS applications

NASA's Aviation Safety Program, Synthetic Vision Systems Project is conducting research in advanced flight deck concepts, such as Synthetic/Enhanced Vision Systems (S/EVS), for commercial and business aircraft. An emerging thrust in this activity is the development of spatially-integrated, large field-of-regard information display systems. Head-worn or helmet-mounted display systems are being proposed as one method in which to meet this objective. System delays or latencies inherent to spatially-integrated, head-worn displays critically influence the display utility, usability, and acceptability. Research results from three different, yet similar technical areas - flight control, flight simulation, and virtual reality - are collectively assembled in this paper to create a global perspective of delay or latency effects in head-worn or helmet-mounted display systems. Consistent definitions and measurement techniques are proposed herein for universal application and latency requirements for Head-Worn Display S/EVS applications are drafted. Future research areas are defined.

[1]  Subbarayan Pasupathy,et al.  Predictive head movement tracking using a Kalman filter , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[2]  Randall E. Bailey,et al.  CFIT prevention using synthetic vision , 2003, SPIE Defense + Commercial Sensing.

[3]  Randall Bailey,et al.  EFFECT OF TIME DELAY ON MANUAL FLIGHT CONTROL AND FLYING QUALITIES DURING IN-FLIGHT AND GROUND-BASED SIMULATION , 1987 .

[4]  Matthias M. Wloka.  Lag in Multiprocessor Virtual Reality , 1995, Presence: Teleoperators & Virtual Environments.

[5]  Randall E. Bailey,et al.  Flight test evaluation of tactical synthetic vision display concepts in a terrain-challenged operating environment , 2002, SPIE Defense + Commercial Sensing.

[6]  George A. Bekey,et al.  Compensation for end to end delays in a VR system , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[7]  Donald C. Wunsch,et al.  Predictive head tracking for virtual reality , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[8]  Susumu Tachi,et al.  Compensation of time lag between actual and virtual spaces by multi-sensor integration , 1994, Proceedings of 1994 IEEE International Conference on MFI '94. Multisensor Fusion and Integration for Intelligent Systems.

[9]  Henry Been-Lirn Duh,et al.  Effects of field of view on presence, enjoyment, memory, and simulator sickness in a virtual environment , 2002, Proceedings IEEE Virtual Reality 2002.

[10]  Arthur W. Gubbels,et al.  In-flight evaluation of a fiber optic helmet-mounted display , 1998, Defense, Security, and Sensing.

[11]  Mark R. Mine.  Characterization of End-to-End Delays in Head-Mounted Display Systems , 1993 .

[12]  Michael Venturino,et al.  Performance and head movements using a helmet-mounted display with different sized fields-of-view , 1990 .

[13]  Richard H. Y. So,et al.  Experimental studies of the use of phase lead filters to compensate lags in head-coupled visual displays , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[14]  Robert S. Bolia,et al.  ASSESSING SIMULATOR SICKNESS IN A SEE-THROUGH HMD: EFFECTS OF TIME DELAY, TIME ON TASK, AND TASK COMPLEXITY , 2000 .

[15]  Scott S. Fisher,et al.  Defining, modeling, and measuring system lag in virtual environments , 1990, Other Conferences.

[16]  PauschRandy,et al.  A literature survey for virtual environments , 1992 .

[17]  Randall E. Bailey.  Effect of head-up display dynamics on fighter flying qualities , 1989 .

[18]  Gary Bishop,et al.  Latency meter: a device for easily monitoring VE delay , 2002 .

[19]  Bernard D. Adelstein,et al.  Improved temporal response in virtual environments through system hardware and software reorganization , 1996, Electronic Imaging.

[20]  R. M. Smith.  A method for determining transport delays in the flight simulation environment , 1991 .

[21]  Jae Y. Jurg.  DISCRIMINABILITY OF PREDICTION ARTIFACTS IN A TIME-DELAYED VIRTUAL ENVIRONMENT , 1999 .

[22]  M. Whitton,et al.  Effect of latency on presence in stressful virtual environments , 2003, IEEE Virtual Reality, 2003. Proceedings..

[23]  Robert S. Allison,et al.  Tolerance of temporal delay in virtual environments , 2001, Proceedings IEEE Virtual Reality 2001.

[24]  Scott J Horowitz.  Measurement and Effects of Transport Delays in a State-of-the-Art F-16C Flight Simulator. , 1987 .

[25]  Bernard D. Adelstein,et al.  Factors influencing operator interaction with virtual objects viewed via head-mounted see-through displays: viewing conditions and rendering latency , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[26]  Jan B.F. van Erp,et al.  Driving with a Head-Slaved Camera System , 2001 .

[27]  Bernard D. Adelstein,et al.  Dynamic Response of Electromagnetic Spatial Displacement Trackers , 1996, Presence: Teleoperators & Virtual Environments.

[28]  R. Galloway,et al.  Simulation cue validation using frequency response techniques , 1996 .

[29]  R.J.A.W. Hosman,et al.  Visual-Vestibular Interactions in the Perception and Control of Aircraft Motions by the Pilot , 1997 .

[30]  Mark A. Livingston,et al.  Managing latency in complex augmented reality systems , 1997, SI3D.

[31]  Randall E. Bailey,et al.  Synthetic Vision Enhances Situation Awareness and RNP Capabilities for Terrain-Challenged Approaches , 2003 .

[32]  Joel S. Warm,et al.  Compensation for the effects of time delay in a helmet-mounted display: perceptual adaptation versus algorithmic prediction , 1995, Defense, Security, and Sensing.

[33]  Sion Jennings,et al.  Hybrid enhanced and synthetic vision system architecture for rotorcraft operations , 2002, SPIE Defense + Commercial Sensing.

[34]  Clarence E. Rash,et al.  Helmet-Mounted Displays: Design Issues for Rotary-Wing Aircraft , 2001 .

[35]  Mark H. Draper,et al.  Effects of Image Scale and System Time Delay on Simulator Sickness within Head-Coupled Virtual Environments , 2001, Hum. Factors.

[36]  Richard L. Newman,et al.  HMD symbol stabilization concepts , 1995, Defense, Security, and Sensing.

[37]  Gary E. Riccio,et al.  The Effects of Simulator Delays on the Acquisition of Flight Control Skills: Control of Heading and Altitude , 1987 .

[38]  Yaron Kranz.  Latency: requirements and implementation of a helmet-mounted cueing system (HMCS) in a fast jet , 1999, Defense, Security, and Sensing.

[39]  Michael J. Griffin,et al.  Effects of time delays on head tracking performance and the benefits of lag compensation by image deflection , 1991 .

[40]  Randy F. Pausch,et al.  A Literature Survey for Virtual Environments: Military Flight Simulator Visual Systems and Simulator Sickness , 1992, Presence: Teleoperators & Virtual Environments.

[41]  Ronald B. Ewart,et al.  An Electronic Visual Display Attitude Sensor (EVDAS) for analysis offlight simulator delays , 1992 .

[42]  Richard E. Mcfarland.  CGI Delay Compensation , 1986 .