Effects of visual and haptic latency on touchscreen interaction: A case study using painting task

This paper reports a user study on the effects of latency in visual and haptic feedback on touchscreen interaction for a painting task. Our work was motivated by recently emerging multimodal use of touchscreens and electrostatic friction displays with high-quality 3D graphics. We designed and implemented a painting application on a touchscreen that enabled users to paint a 3D sculpture with their finger pad while perceiving haptic feedback through electrovibration. Software-induced latency was varied from 0 to 120 ms for both visual and haptic feedback. Participants' task was to paint on the 3D sculpture as quickly and accurately as possible. For performance assessment, we measured task completion time and execution error. We also obtained subjective responses to four questions (easiness, responsiveness, pleasantness, and the sense of control) related to user experiences. Experiment results indicated that visual latency is critical for both task completion time and task execution error whereas haptic latency is for task execution error, but not for task completion time. Both latencies affected the subjective responses, but visual latency had more apparent effects.

[1]  In Lee,et al.  Discrimination of visual and haptic rendering delays in networked environments , 2009 .

[2]  John Quarles,et al.  Latency and avatars in Virtual Environments and the effects on gait for persons with mobility impairments , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[3]  Olivier Bau,et al.  REVEL: tactile feedback technology for augmented reality , 2012, ACM Trans. Graph..

[4]  M. Jeannerod.  The mechanism of self-recognition in humans , 2003, Behavioural Brain Research.

[5]  Eve E. Hoggan,et al.  The effect of tactile feedback latency in touchscreen interaction , 2011, 2011 IEEE World Haptics Conference.

[6]  Marc Erich Latoschik,et al.  Avatar embodiment realism and virtual fitness training , 2015, 2015 IEEE Virtual Reality (VR).

[7]  E MALLINCKRODT,et al.  Perception by the skin of electrically induced vibrations. , 1953, Science.

[8]  Azzedine Boukerche,et al.  The Effect of Prediction on Collaborative Haptic Applications , 2006, 2006 14th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[9]  Jee-Hwan Ryu,et al.  Performance analysis of teleoperation systems with different haptic and video time-delay , 2009, 2009 ICCAS-SICE.

[10]  Ju-Hwan Lee,et al.  Spatiotemporal Visuotactile Interaction , 2008, EuroHaptics.

[11]  Ali Israr,et al.  TeslaTouch: electrovibration for touch surfaces , 2010, UIST.

[12]  Ingrid M. L. C. Vogels,et al.  Detection of Temporal Delays in Visual-Haptic Interfaces , 2004, Hum. Factors.

[13]  Caroline Jay,et al.  Delayed visual and haptic feedback in a reciprocal tapping task , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[14]  Seung-Chan Kim,et al.  Tactile rendering of 3D features on touch surfaces , 2013, UIST.

[15]  Donald E. Troxel,et al.  An Electrotactile Display , 1970 .

[16]  Stefan Kopp,et al.  The impact of latency on perceptual judgments and motor performance in closed-loop interaction in virtual reality , 2016, VRST.

[17]  Mar González-Franco,et al.  The contribution of real-time mirror reflections of motor actions on virtual body ownership in an immersive virtual environment , 2010, 2010 IEEE Virtual Reality Conference (VR).

[18]  Seungmoon Choi,et al.  Identification of primitive geometrical shapes rendered using electrostatic friction display , 2016, 2016 IEEE Haptics Symposium (HAPTICS).

[19]  J. Edward Colgate,et al.  Dynamics of ultrasonic and electrostatic friction modulation for rendering texture on haptic surfaces , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[20]  J. Edward Colgate,et al.  Fingertip friction modulation due to electrostatic attraction , 2013, 2013 World Haptics Conference (WHC).

[21]  Caroline Jay,et al.  Modeling the effects of delayed haptic and visual feedback in a collaborative virtual environment , 2007, TCHI.

[22]  I. Scott MacKenzie,et al.  Lag as a determinant of human performance in interactive systems , 1993, INTERCHI.

[23]  Robert S. Allison,et al.  Effects of network delay on a collaborative motor task with telehaptic and televisual feedback , 2004, VRCAI '04.

[24]  Eve E. Hoggan,et al.  Playing with Tactile Feedback Latency in Touchscreen Interaction: Two Approaches , 2011, INTERACT.