Effects of Stereoscopic Viewing and Haptic Feedback, Sensory-Motor Congruence and Calibration on Near-Field Fine Motor Perception-Action Coordination in Virtual Reality

We present an empirical evaluation on how stereoscopic viewing and haptic feedback deferentially affects fine motor perception-action coordination in a pick-and-place task in Virtual Reality (VR). The factors considered were stereoscopic viewing, haptic feedback, sensory-motor congruence and mismatch, and calibration on perception-action coordination in near field fine motor task performance in VR. Quantitative measures of placement error, distance, collision, and time to complete trials were recorded and analyzed. Overall, we found that participants' manual dexterous task performance was enhanced in the presence of both stereoscopic viewing and haptic feedback. However, we found that time to complete task was greatly enhanced by the presence of haptic feedback, and economy and efficiency of movement of the end effector as well as the manipulated object was enhanced by the presence of both haptic feedback and stereoscopic viewing. Whereas, number of collisions and placement accuracy were greatly enhanced by the presence of stereoscopic viewing in near-field fine motor perception-action coordination. Our research additionally shows that mismatch in sensory-motor stimuli can detrimentally affect the number of collisions, and efficiency of end effector and object movements in near-field fine motor activities, and can be further negatively affected by the absence of haptic feedback and stereoscopic viewing. In spite of reduced cue situations in VR, and the absence or presence of stereoscopic viewing and haptic feedback, we found that participants tend to calibrate or adapt their perception-action coordination rapidly with a set of at least 5 trials.

[1]  John Kenneth Salisbury,et al.  Haptic Feedback Enhances Force Skill Learning , 2007, Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC'07).

[2]  Mark Wright,et al.  The Effect of Haptic Feedback and Stereo Graphics in a 3D Target Acquisition Task , 2002 .

[3]  Vassilis Cutsuridis,et al.  Cognitive models of the perception-action cycle: A view from the brain , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[4]  M. Posner,et al.  Visual dominance: an information-processing account of its origins and significance. , 1976, Psychological review.

[5]  Gerd Bruder,et al.  Analysis of direct selection in head-mounted display environments , 2014, 2014 IEEE Symposium on 3D User Interfaces (3DUI).

[6]  Gerald M. Fried,et al.  FLS Assessment of Competency Using Simulated Laparoscopic Tasks , 2008, Journal of Gastrointestinal Surgery.

[7]  William Ribarsky,et al.  Balancing fusion, image depth and distortion in stereoscopic head-tracked displays , 1999, SIGGRAPH.

[8]  Frans C. T. van der Helm,et al.  Influence of haptic guidance in learning a novel visuomotor task , 2009, Journal of Physiology-Paris.

[9]  Lucian Panait,et al.  The role of haptic feedback in laparoscopic simulation training. , 2009, The Journal of surgical research.

[10]  M. Žefran,et al.  Assessment of faculty perception of content validity of PerioSim, a haptic-3D virtual reality dental training simulator. , 2007, Journal of dental education.

[11]  James L. Patton,et al.  A Real-Time Haptic/Graphic Demonstration of how Error Augmentation can Enhance Learning , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[12]  James E. Cutting,et al.  Chapter 3 – Perceiving Layout and Knowing Distances: The Integration, Relative Potency, and Contextual Use of Different Information about Depth* , 1995 .

[13]  Farid Taha,et al.  Specification and design of a new haptic interface for maxillo facial surgery , 2011, 2011 IEEE International Conference on Robotics and Automation.

[14]  Alejandro Jarillo Silva,et al.  PHANToM OMNI Haptic Device: Kinematic and Manipulability , 2009, 2009 Electronics, Robotics and Automotive Mechanics Conference (CERMA).

[15]  Charles E. Pettypiece,et al.  Integration of haptic and visual size cues in perception and action revealed through cross-modal conflict , 2010, Experimental Brain Research.

[16]  G. Andriole,et al.  2D versus 3D visualization: impact on laparoscopic proficiency using the fundamentals of laparoscopic surgery skill set. , 2012, Journal of laparoendoscopic & advanced surgical techniques. Part A.

[17]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[18]  Bliss M. Altenhoff,et al.  Near-field distance perception in real and virtual environments using both verbal and action responses , 2011, TAP.

[19]  Robert B. Welch,et al.  The “ventriloquist effect”: Visual dominance or response bias? , 1975 .

[20]  Betty J. Mohler,et al.  The influence of feedback on egocentric distance judgments in real and virtual environments , 2006, APGV '06.

[21]  R. Klatzky,et al.  Haptic perception: A tutorial , 2009, Attention, perception & psychophysics.

[22]  K. Moorthy,et al.  The benfits of stereoscopic vision in robotic-assisted performance on bench models , 2004, Surgical Endoscopy And Other Interventional Techniques.

[23]  Heather Culbertson,et al.  Importance of Matching Physical Friction, Hardness, and Texture in Creating Realistic Haptic Virtual Surfaces , 2017, IEEE Transactions on Haptics.

[24]  Frank T. J. M. Zaal,et al.  Virtual Reality as a Tool for the Study of Perception-Action: The Case of Running to Catch Fly Balls , 2011, PRESENCE: Teleoperators and Virtual Environments.

[25]  Gerd Bruder,et al.  Distance estimation in large immersive projection systems, revisited , 2015, 2015 IEEE Virtual Reality (VR).

[26]  B. Alaraimi,et al.  A Randomized Prospective Study Comparing Acquisition of Laparoscopic Skills in Three-Dimensional (3D) vs. Two-Dimensional (2D) Laparoscopy , 2014, World Journal of Surgery.

[27]  J. Fuster.  Upper processing stages of the perception–action cycle , 2004, Trends in Cognitive Sciences.

[28]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[29]  Serban R. Pop,et al.  The Use of Stereoscopy in a Neurosurgery Training Virtual Environment , 2016, PRESENCE: Teleoperators and Virtual Environments.

[30]  Ryan P. McMahan.  Exploring the Effects of Higher-Fidelity Display and Interaction for Virtual Reality Games , 2011 .

[31]  Vincent Hayward,et al.  Performance Measures for Haptic Interfaces , 1996 .

[32]  Robert J. Teather,et al.  Pointing at 3D targets in a stereo head-tracked virtual environment , 2011, 2011 IEEE Symposium on 3D User Interfaces (3DUI).

[33]  Domenico Prattichizzo,et al.  Simplified design of haptic display by extending one-point kinesthetic feedback to multipoint tactile feedback , 2010, 2010 IEEE Haptics Symposium.

[34]  Christopher C. Pagano,et al.  An Empirical Evaluation of Visuo-Haptic Feedback on Physical Reaching Behaviors During 3D Interaction in Real and Immersive Virtual Environments , 2016, ACM Trans. Appl. Percept..

[35]  Matthieu Poyade,et al.  Validation of a haptic virtual reality simulation in the context of industrial maintenance , 2012 .

[36]  Melina C Vassiliou,et al.  Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room-a randomized controlled trial. , 2010, American journal of surgery.

[37]  Blake Hannaford,et al.  Control law design for haptic interfaces to virtual reality , 2002, IEEE Trans. Control. Syst. Technol..

[38]  G P Bingham,et al.  Comparing measures of monocular distance perception: verbal and reaching errors are not correlated. , 1998, Journal of experimental psychology. Human perception and performance.

[39]  Matthieu Poyade,et al.  Haptic Plug-In For Unity3D , 2014 .

[40]  Ling Guan,et al.  An Approach to Ballet Dance Training through MS Kinect and Visualization in a CAVE Virtual Reality Environment , 2015, ACM Trans. Intell. Syst. Technol..

[41]  Darwin G. Caldwell,et al.  Integrating Haptics with Augmented Reality in a Femoral Palpation and Needle Insertion Training Simulation , 2011, IEEE Transactions on Haptics.

[42]  Hermann J. Müller,et al.  Influence of visuomotor action on visual-haptic simultaneous perception: A psychophysical study , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[43]  Luv Kohli,et al.  Exploiting Perceptual Illusions to Enhance Passive Haptics , 2009 .

[44]  Larry F. Hodges,et al.  A schematic eye for virtual environments , 2016, 2016 IEEE Symposium on 3D User Interfaces (3DUI).