Effects of Packet Loss and Latency on the Temporal Discrimination of Visual-Haptic Events

Temporal discontinuities and delay caused by packet loss or communication latency often occur in multimodal telepresence systems. It is known that such artifacts can influence the feeling of presence [1]. However, it is largely unknown how the packet loss and communication latency affect the temporal perception of multisensory events. In this article, we simulated random packet dropouts and communication latency in the visual modality and investigated the effects on the temporal discrimination of visual-haptic collisions. Our results demonstrated that the synchronous perception of crossmodal events was very sensitive to the packet loss rate. The packet loss caused the impression of time delay and influenced the perception of the subsequent events. The perceived time of the visual event increased linearly, and the temporal discrimination deteriorated, with increasing packet loss rate. The perceived time was also influenced by the communication delay, which caused time to be slightly overestimated.

[1]  Jan B F van Erp,et al.  Vibro-Tactile and Visual Asynchronies: Sensitivity and Consistency , 2004, Perception.

[2]  W R Ferrell.  Delayed Force Feedback1 , 1966, Human factors.

[3]  Jonathan Steuer,et al.  Defining virtual reality: dimensions determining telepresence , 1992 .

[4]  Donald F. Towsley,et al.  Measurement and modelling of the temporal dependence in packet loss , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[5]  Georg Färber,et al.  Photo-realistic scene prediction of partially unknown environments for the compensation of time delays in telepresence applications , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[6]  S. Ellis.  Pictorial communication in virtual and real environments , 1991 .

[7]  Jaideep Srivastava,et al.  Experimental evaluation of loss perception in continuous media , 1999, Multimedia Systems.

[8]  Gerd Hirzinger,et al.  Sensor-based space robotics-ROTEX and its telerobotic features , 1993, IEEE Trans. Robotics Autom..

[9]  Bernard D. Adelstein,et al.  Head Tracking Latency in Virtual Environments: Psychophysics and a Model , 2003 .

[10]  Mark Claypool,et al.  The effects of loss and latency on user performance in unreal tournament 2003® , 2004, NetGames '04.

[11]  Thomas B. Sheridan,et al.  Remote Manipulative Control with Transmission Delay , 1963 .

[12]  E. Gilbert.  Capacity of a burst-noise channel , 1960 .

[13]  Hermann J. Müller,et al.  Influence of visuomotor action on visual-haptic simultaneous perception: A psychophysical study , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[14]  Sandra Hirche,et al.  Human Perceived Transparency with Time Delay , 2007, Advances in Telerobotics.

[15]  D. Collett,et al.  Modelling Binary Data , 1991 .

[16]  Karon E. MacLean,et al.  The perception of cross-modal simultaneity (or ``the Greenwich Observatory Problem'' revisited) , 2001 .

[17]  H. Kalmus,et al.  Effects of Delayed Visual Control on Writing, Drawing and Tracing , 1960 .

[18]  I. Scott MacKenzie,et al.  Lag as a determinant of human performance in interactive systems , 1993, INTERCHI.

[19]  Mark Claypool,et al.  The effects of jitter on the peceptual quality of video , 1999, MULTIMEDIA '99.

[20]  Lars C. Wolf,et al.  Analysis of factors affecting players' performance and perception in multiplayer games , 2005, NetGames '05.

[21]  David Hands,et al.  A Study of the Impact of Network Loss and Burst Size on Video Streaming Quality and Acceptability , 1999, IDMS.

[22]  Zheng Wang,et al.  Internet QoS: Architectures and Mechanisms for Quality of Service , 2001 .

[23]  James V. Stone,et al.  When is now? Perception of simultaneity , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[24]  E. O. Elliott.  A model of the switched telephone network for data communications , 1965 .

[25]  K U Smith,et al.  Delayed Visual Feedback and Behavior , 1960, Science.

[26]  Nathaniel I. Durlach,et al.  Telepresence, time delay and adaptation , 1991 .

[27]  Caroline Jay,et al.  Modeling the effects of delayed haptic and visual feedback in a collaborative virtual environment , 2007, TCHI.

[28]  Charles Spence,et al.  Evaluating the influence of frame rate on the temporal aspects of audiovisual speech perception , 2006, Neuroscience Letters.

[29]  N. F. Dixon,et al.  The Detection of Auditory Visual Desynchrony , 1980, Perception.

[30]  Daniel J. Levitino,et al.  The Perception of Cross-Modal Simultaneity , 2001 .

[31]  Konrad Paul Kording,et al.  Review TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Bayesian decision theory in sensorimotor control , 2022 .

[32]  Henning Sanneck,et al.  Packet loss recovery and control for voice transmission over the Internet , 2000 .

[33]  C. Spence,et al.  Multisensory prior entry. , 2001, Journal of experimental psychology. General.

[34]  C. Spence,et al.  Spatial constraints on visual-tactile cross-modal distractor congruency effects , 2004, Cognitive, affective & behavioral neuroscience.