Impact of communication delay and temporal sensitivity on perceived workload and teleoperation performance

As teleoperated robotic units operate in remote and distant environments such as outer space, the communication latency impacts the operator’s performance. Yet, the delay impact on teleoperation performance and mental workload has scarcely been evaluated. Human temporal sensitivity appears to modulate the impact of latency on operators’ performance but no joint assessment of temporal sensitivity and subjective workload has been reported. In this study, we assess the relationship between the impact of communication delay on teleoperation performance, mental workload, and operators’ temporal sensitivity. Sixteen participants completed two online tasks: a duration reproduction task in which they were asked to reproduce the duration of previously presented visual stimuli, and an egocentric maze navigation task which required participants to escape a static maze, under an input latency of 0, 400, and 3000 ms. Completion time, move count, and error rate were recorded for each trial, along with perceived workload using the NASA-TLX questionnaire. The results showed that performance was significantly deteriorated by an increase in communication delay. Moreover, participants’ self-rated performance decreased with a larger communication delay, while their reported frustration, effort, and mental demands significantly increased. Interestingly, a possible effect of the temporal sensitivity profile on teleoperation performance - number of moves - was found, with a reduced number of moves for sensitive participants compared to insensitive ones, following a speed/accuracy trade-off (yet not significant). Hence, different operators’ strategies were uncovered, depending on their temporal sensitivity profile, to mitigate the impact of communication delay on the mission outcome.

[1]  I. Kozlovskaya,et al.  Slowing of human arm movements during weightlessness: the role of vision , 2002, European Journal of Applied Physiology.

[2]  Wim Lamotte,et al.  Influence of network delay and jitter on cooperation in multiplayer games , 2011, VRCAI '11.

[3]  David B. Kaber,et al.  Effects of Visual Interface Design, and Control Mode and Latency on Performance, Telepresence and Workload in a Teleoperation Task , 2000 .

[4]  Bernard D. Adelstein,et al.  Sensor spatial distortion, visual latency, and update rate effects on 3D tracking in virtual environments , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[5]  Kohei Takeda,et al.  Survey in Fukushima Daiichi NPS by combination of human and remotely-controlled robot , 2017, 2017 IEEE International Symposium on Safety, Security and Rescue Robotics (SSRR).

[6]  Carole Ferrel,et al.  Contrôle sensori-moteur en situation de téléopération : données théoriques et perspectives ergonomiques , 2002 .

[7]  W. Kuhmann Experimental investigation of stress-inducing properties of system response times. , 1989, Ergonomics.

[8]  David L. Akin,et al.  Human factors optimization of virtual environment attributes for a space telerobotic control station , 2000 .

[9]  W Kuhmann,et al.  Experimental investigation of psychophysiological stress-reactions induced by different system response times in human-computer interaction. , 1987, Ergonomics.

[10]  G. Stoet PsyToolkit , 2017 .

[11]  W. Boucsein,et al.  Standardized task strain and system response times in human-computer interaction. , 1995, Ergonomics.

[12]  William R. Ferrell,et al.  Remote manipulation with transmission delay. , 1965 .

[13]  I. Scott MacKenzie,et al.  Lag as a determinant of human performance in interactive systems , 1993, INTERCHI.

[14]  Freek Stulp,et al.  Teleoperating Robots from the International Space Station: Microgravity Effects on Performance with Force Feedback , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[15]  Charles G. Halcomb,et al.  Time in human-computer interaction: performance as a function of delay type, delay duration, and task difficulty , 1990 .

[16]  Barbara S. Chaparro,et al.  The world wide wait: effects of delays on user performance , 2000 .

[17]  Kellogg S. Booth,et al.  Evaluating 3D task performance for fish tank virtual worlds , 1993, TOIS.

[18]  Kaleb McDowell,et al.  The Effects of Time Lag on Driving Performance and a Possible Mitigation , 2010, IEEE Transactions on Robotics.

[19]  Robert S. Bolia,et al.  ASSESSING SIMULATOR SICKNESS IN A SEE-THROUGH HMD: EFFECTS OF TIME DELAY, TIME ON TASK, AND TASK COMPLEXITY , 2000 .

[20]  Lars C. Wolf,et al.  On the impact of delay on real-time multiplayer games , 2002, NOSSDAV '02.

[21]  Gijsbert Stoet,et al.  PsyToolkit: A Novel Web-Based Method for Running Online Questionnaires and Reaction-Time Experiments , 2017 .

[22]  Thibault Gateau,et al.  How Can Physiological Computing Benefit Human-Robot Interaction? , 2020, Robotics.

[23]  Craig R. Carignan,et al.  Effects of time delay on telerobotic control of neutral buoyancy vehicles , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[24]  Sophie Jörg,et al.  Player perception of delays and jitter in character responsiveness , 2014, SAP.

[25]  I Kozlovskaya,et al.  Pointing arm movements in short- and long-term spaceflights. , 1997, Aviation, space, and environmental medicine.

[26]  R. Held,et al.  Adaptation to displaced and delayed visual feedback from the hand. , 1966 .

[27]  Thomas B. Sheridan,et al.  Space teleoperation through time delay: review and prognosis , 1993, IEEE Trans. Robotics Autom..

[28]  L. E. Bourne,et al.  Effects of delay of information feedback and task complexity on the identification of concepts. , 1957, Journal of experimental psychology.

[29]  Charles R Doarn,et al.  Evaluation of surgical skills in microgravity using force sensing. , 2005, Medical engineering & physics.

[30]  W W Wierwille,et al.  Effects of Visual Display and Motion System Delays on Operator Performance and Uneasiness in a Driving Simulator , 1988, Human factors.

[31]  L E Marks,et al.  Cross-modality matching functions generated by magnitude estimation , 1980, Perception & psychophysics.

[32]  Kajal T. Claypool,et al.  Latency and player actions in online games , 2006, CACM.

[33]  Douglas J. Gillan,et al.  Using Temporal Sensitivity to Predict Performance Under Latency in Teleoperation , 2018, Hum. Factors.