Examining the Effects of Interaction Fidelity on Task Performance and Learning in Virtual Reality

Virtual reality training applications are becoming more widespread as hardware costs for head-mounted displays (HMDs) and tracking technology decrease, enabling a wide variety of interaction techniques beyond the mouse and keyboard. Advancements in VR hardware yield higher levels of interaction fidelity or the objective degree of exactness with which real-world interactions can be reproduced in an interactive system. Despite the growing use of virtual reality simulations for training, few prior investigations have examined the constituent components of fidelity and how they affect performance and learning in the context of a real-world skills training applications. With tracking and HMD technology reaching a wider audience, it is important to empirically evaluate the benefits and downsides associated with utilizing such technology. Using our scaffolded virtual metrology training simulation as a testbed for exploring issues related to interaction fidelity, we have created a bimanual interaction metaphor that allows users to select and manipulate objects with 6 degrees-of-freedom using both hands simultaneously. The simulation guides the user step-by-step on the process of taking precise measurements using calipers and micrometers. Our initial work examined the effects of simplifying interaction by constraining the number of degrees-of-freedom on measures including user performance, cognitive and psychomotor learning outcomes, presence, and perceived workload. Results indicate that the training application was beneficial in teaching basic metrology concepts and simplifying the interaction metaphor resulted in similar user performance in most metrics. Our following experiment examined the effects of presentation method including a HMD with co-located end-effectors compared to a large-screen, immersive display with dislocated endeffectors on user performance and learning outcomes. In addition, physics fidelity was examined by either enabling or disabling gravity to determine usability improvements. Results indicate that the HMD condition was preferable to the immersive display in nearly all metrics while the no-gravity

[1]  Larry F. Hodges,et al.  Taxonomy and experimental evaluation of two-handed selection techniques for volumetric data , 2008 .

[2]  Bliss M. Altenhoff,et al.  Effects of calibration to visual and haptic feedback on near-field depth perception in an immersive virtual environment , 2012, SAP.

[3]  David Swapp,et al.  Interaction with co-located haptic feedback in virtual reality , 2006, Virtual Reality.

[4]  Larry F. Hodges,et al.  Development of a scenario builder tool for scaffolded virtual patients , 2014, 2014 IEEE Virtual Reality (VR).

[5]  Robert W. Lindeman,et al.  Towards usable VR: an empirical study of user interfaces for immersive virtual environments , 1999, CHI '99.

[6]  Yanqing Wang,et al.  Effects of Orientation Disparity Between Haptic and Graphic Displays of Objects in Virtual Environments , 1999, INTERACT.

[7]  Eric D. Ragan,et al.  Studying the Effects of Stereo, Head Tracking, and Field of Regard on a Small-Scale Spatial Judgment Task , 2013, IEEE Transactions on Visualization and Computer Graphics.

[8]  John T. Bell,et al.  Ten steps to developing virtual reality applications for engineering education , 1997 .

[9]  Maureen K. Holden,et al.  Use of Virtual Environments in Motor Learning and Rehabilitation , 2002 .

[10]  Valerie J. Shute,et al.  Intelligent Tutoring Systems: Past, Present, and Future. , 1994 .

[11]  Florence Tyndiuk,et al.  Cognitive comparison of 3D interaction in front of large vs. small displays , 2005, VRST '05.

[12]  Frank Rudzicz,et al.  A framework for 3D visualisation and manipulation in an immersive space using an untethered bimanual gestural interface , 2004, VRST '04.

[13]  Florence Tyndiuk,et al.  Impact of large displays on virtual reality task performance , 2004, AFRIGRAPH '04.

[14]  Mel Slater,et al.  A Framework for Immersive Virtual Environments (FIVE): Speculations on the Role of Presence in Virtual Environments , 1997, Presence: Teleoperators & Virtual Environments.

[15]  A Weckenmann,et al.  MANUFACTURING METROLOGY - STATE OF THE ART AND PROSPECTS , 2007 .

[16]  Niklas Elmqvist,et al.  A Taxonomy of 3D Occlusion Management for Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[17]  Christoph W. Borst,et al.  Visual feedback for virtual grasping , 2014, 2014 IEEE Symposium on 3D User Interfaces (3DUI).

[18]  Doug A. Bowman,et al.  Formalizing the Design, Evaluation, and Application of Interaction Techniques for Immersive Virtual Environments , 1999, J. Vis. Lang. Comput..

[19]  Frederick P. Brooks,et al.  Moving objects in space: exploiting proprioception in virtual-environment interaction , 1997, SIGGRAPH.

[20]  H. Hellwig.  The importance of measurement in technology-based competition , 1990 .

[21]  Colin Ware,et al.  The Importance of Stereo and Eye-Coupled Perspective for Eye-Hand Coordination in Fish Tank VR , 2004, Presence: Teleoperators & Virtual Environments.

[22]  Bernd Fröhlich,et al.  Two-handed direct manipulation on the responsive workbench , 1997, SI3D.

[23]  Frederick P. Brooks.  What's Real About Virtual Reality? , 1999, IEEE Computer Graphics and Applications.

[24]  W. Buxton,et al.  A study in two-handed input , 1986, CHI '86.

[25]  James M. Hennessey,et al.  Comparing single- and two-handed 3D input for a 3D object assembly task , 1998, CHI Conference Summary.

[26]  Dennis Proffitt,et al.  Attention and visual feedback: the bimanual frame of reference , 1997, SI3D.

[27]  Yoshifumi Kitamura,et al.  A manipulation environment of virtual and real objects using a magnetic metaphor , 2002, VRST '02.

[28]  Abdul-Hadi G. Abulrub,et al.  Virtual reality in engineering education: The future of creative learning , 2011, EDUCON 2011.

[29]  Ivan Poupyrev,et al.  The go-go interaction technique: non-linear mapping for direct manipulation in VR , 1996, UIST '96.

[30]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[31]  Kyle Johnsen,et al.  Immersive 3DUI on one dollar a day , 2012, 2012 IEEE Symposium on 3D User Interfaces (3DUI).

[32]  T. Grantcharov,et al.  Randomized clinical trial of virtual reality simulation for laparoscopic skills training , 2004, The British journal of surgery.

[33]  A. Darzi,et al.  Skill transfer from virtual reality to a real laparoscopic task , 2001, Surgical Endoscopy.

[34]  Doug A. Bowman,et al.  Virtual Reality: How Much Immersion Is Enough? , 2007, Computer.

[35]  A. Gallagher,et al.  Objective Psychomotor Skills Assessment of Experienced, Junior, and Novice Laparoscopists with Virtual Reality , 2001, World Journal of Surgery.

[36]  Doug A. Bowman,et al.  An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments , 1997, SI3D.

[37]  Peter J. Werkhoven,et al.  Visuomotor Adaptation to Virtual Hand Position in Interactive Virtual Environments , 1998, Presence.

[38]  Lakhmi C. Jain,et al.  Advanced Computational Intelligence Paradigms in Healthcare 6. Virtual Reality in Psychotherapy, Rehabilitation, and Assessment , 2011 .

[39]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[40]  Doug A. Bowman,et al.  A Tool for the Interactive 3D Visualization of Electronic Structure in Molecules and Solids , 2002, Comput. Chem..

[41]  Joseph J. LaViola,et al.  Dense and Dynamic 3D Selection for Game-Based Virtual Environments , 2012, IEEE Transactions on Visualization and Computer Graphics.

[42]  T. Ichikawa,et al.  Egocentric Object Manipulation in Virtual Environments: Empirical Evaluation of Interaction Techniques , 1998, Comput. Graph. Forum.

[43]  Karl F. MacDorman,et al.  The Uncanny Valley [From the Field] , 2012, IEEE Robotics Autom. Mag..

[44]  Ravin Balakrishnan,et al.  Exploring bimanual camera control and object manipulation in 3D graphics interfaces , 1999, CHI '99.

[45]  B. Bloom,et al.  Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , 1966 .

[46]  Teresa Gutiérrez,et al.  IMA-VR: A multimodal virtual training system for skills transfer in Industrial Maintenance and Assembly tasks , 2010, 19th International Symposium in Robot and Human Interactive Communication.

[47]  Doug A. Bowman,et al.  The benefits of immersion for spatial understanding of complex underground cave systems , 2007, VRST '07.

[48]  Kaushik Das,et al.  An evaluation of menu properties and pointing techniques in a projection-based VR environment , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[49]  Charles L. A. Clarke,et al.  symSpline: symmetric two-handed spline manipulation , 2006, CHI.

[50]  Robert J. Armstrong.  Developing and Writing Behavioral Objectives. , 1970 .

[51]  Jeffrey W. Bertrand,et al.  The role of dimensional symmetry on bimanual psychomotor skills education in immersive virtual environments , 2015, 2015 IEEE Virtual Reality (VR).

[52]  Benjamin Lok,et al.  Scaffolded learning with mixed reality , 2009, Comput. Graph..

[53]  Jim X. Chen,et al.  A Model for Understanding How Virtual Reality Aids Complex Conceptual Learning , 1999, Presence: Teleoperators & Virtual Environments.

[54]  Douglas A. Bowman,et al.  Interaction Techniques For Common Tasks In Immersive Virtual Environments - Design, Evaluation, And , 1999 .

[55]  Ken Hinckley,et al.  Passive real-world interface props for neurosurgical visualization , 1994, International Conference on Human Factors in Computing Systems.

[56]  Amy Banic,et al.  Two Handed Selection Techniques for Volumetric Data , 2007, 2007 IEEE Symposium on 3D User Interfaces.

[57]  Tassos A. Mikropoulos,et al.  Presence: a unique characteristic in educational virtual environments , 2006, Virtual Reality.

[58]  Sharif Razzaque,et al.  The hand is slower than the eye: a quantitative exploration of visual dominance over proprioception , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[59]  Terrence Fernando,et al.  An Immersive Assembly and Maintenance Simulation Environment , 2004, Eighth IEEE International Symposium on Distributed Simulation and Real-Time Applications.

[60]  Fahad Al-Zahrani.  Web-Based Learning and Training for Virtual Metrology Lab , 2010, ArXiv.

[61]  Dennis Proffitt,et al.  Two-handed virtual manipulation , 1998, TCHI.

[62]  Jock D. Mackinlay,et al.  The design space of input devices , 1990, CHI '90.

[63]  A. Darzi,et al.  Curriculum-based solo virtual reality training for laparoscopic intracorporeal knot tying: objective assessment of the transfer of skill from virtual reality to reality. , 2007, American journal of surgery.

[64]  Grigore C. Burdea,et al.  Repetitive bimanual integrative therapy (RABIT) virtual rehabilitation system , 2013, 2013 IEEE Virtual Reality (VR).

[65]  Patrick C. Kyllonen,et al.  Organization and Components of Psychomotor Ability , 2000, Cognitive Psychology.

[66]  Murat Cenk Cavusoglu,et al.  Effect of visuo-haptic co-location on 3D Fitts' task performance , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[67]  Daniel Vogel,et al.  The Impact of Control-Display Gain on User Performance in Pointing Tasks , 2008, Hum. Comput. Interact..

[68]  Joseph J. LaViola,et al.  Poster: Evaluation of menu techniques using a 3D game input device , 2009, 2009 IEEE Symposium on 3D User Interfaces.

[69]  Shumin Zhai,et al.  Manual and cognitive benefits of two-handed input: an experimental study , 1998, TCHI.

[70]  Shoichi Hasegawa,et al.  SPIDAR G&G: A Two-Handed Haptic Interface for Bimanual VR Interaction , 2004 .

[71]  Doug A. Bowman,et al.  Evaluating natural interaction techniques in video games , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[72]  Mel Slater,et al.  Depth of Presence in Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[73]  Randall B. Smith.  Experiences with the Alternate Reality Kit: An Example of the Tension between Literalism and Magic , 1987, IEEE Computer Graphics and Applications.

[74]  Robert S. Allison,et al.  Evaluating visual/motor co-location in fish-tank virtual reality , 2009, 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH).

[75]  Georg Bretthauer,et al.  A Prospective Randomized Study to Test the Transfer of Basic Psychomotor Skills From Virtual Reality to Physical Reality in a Comparable Training Setting , 2005, Annals of surgery.

[76]  Ryan P. McMahan.  Exploring the Effects of Higher-Fidelity Display and Interaction for Virtual Reality Games , 2011 .

[77]  Ken Hinckley,et al.  A survey of design issues in spatial input , 1994, UIST '94.

[78]  Robert Kooima.  Generalized Perspective Projection , 2011 .

[79]  James R. Lewis,et al.  IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use , 1995, Int. J. Hum. Comput. Interact..

[80]  Ken Hinckley,et al.  Haptic issues for virtual manipulation , 1997 .

[81]  M S Wilson,et al.  MIST VR: a virtual reality trainer for laparoscopic surgery assesses performance. , 1997, Annals of the Royal College of Surgeons of England.

[82]  Cullen D. Jackson,et al.  CAVE and fishtank virtual-reality displays: a qualitative and quantitative comparison , 2006, IEEE Transactions on Visualization and Computer Graphics.

[83]  Doug A. Bowman,et al.  Comparing the performance of natural, semi-natural, and non-natural locomotion techniques in virtual reality , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[84]  Robert J Stone,et al.  The (human) science of medical virtual learning environments , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[85]  Ferran Argelaguet,et al.  A survey of 3D object selection techniques for virtual environments , 2013, Comput. Graph..

[86]  Donald B. Johnson,et al.  Testbed Evaluation of Virtual Environment Interaction Techniques , 1999, Presence: Teleoperators & Virtual Environments.

[87]  John S. Lamancusa,et al.  The Learning Factory—A New Approach to Integrating Design and Manufacturing into the Engineering Curriculum , 1997 .

[88]  Hugh Jack.  2013 State of Manufacturing Education , 2013 .

[89]  Shumin Zhai,et al.  Hand eye coordination patterns in target selection , 2000, ETRA.

[90]  Kellogg S. Booth,et al.  Fish tank virtual reality , 1993, INTERCHI.

[91]  Y. Guiard.  Asymmetric division of labor in human skilled bimanual action: the kinematic chain as a model. , 1987, Journal of motor behavior.

[92]  M. Oermann.  Psychomotor skill development. , 1990, Journal of continuing education in nursing.

[93]  Wayne S. Zimmerman,et al.  The Guilford-Zimmerman Aptitude Survey. , 1948 .

[94]  Albert A. Rizzo,et al.  Towards pervasive physical rehabilitation using Microsoft Kinect , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[95]  Arun Yoganandan,et al.  Comparison of a two-handed interface to a wand interface and a mouse interface for fundamental 3D tasks , 2012, 2012 IEEE Symposium on 3D User Interfaces (3DUI).

[96]  Bruno Arnaldi,et al.  GVT: a platform to create virtual environments for procedural training , 2008, 2008 IEEE Virtual Reality Conference.

[97]  S. J. Steiner,et al.  Virtual Reality: A Tool for Assembly? , 2000, Presence: Teleoperators & Virtual Environments.

[98]  Bliss M. Altenhoff,et al.  Effects of visual and proprioceptive information in visuo-motor calibration during a closed-loop physical reach task in immersive virtual environments , 2014, SAP.