Systematic evaluation of a virtual environment-based training systems

This paper explores the application of three constructs that deemed to be essential to quantify virtual environments (VE) efficacy: cognitive, skill-based, and affective learning outcomes. The authors discuss the implementation of these constructs in a user-centered evaluation of a VE training system. By transforming both the conceptual and operational cohorts for training evaluation the authors illustrate the benefits of the development of a Multi-dimensional User-centered Systematic Training Evaluation (MUSTe) method for quantifying VEs efficacy. Importantly, MUSTe acknowledges the importance of combining holistic and analytical approaches in conducting systematic user-based evaluation. Furthermore, it also emphasizes that quantifying VEs efficacy must reflect the perception and preferences of the users rather than the imposition of efficacy on single measures of task outcome. An empirical study that applied MUSTe evaluation method in quantifying a VE training system efficacy provided valuable evidence of the theoretical construct and content validity of the method.

[1]  A. Matthews.  Evaluation Methodology Basics: The Nuts and Bolts of Sound Evaluation , 2006 .

[2]  Saeid Nahavandi,et al.  Computer-simulated Environment (VE) for training : challenge of efficacy evaluation , 2008 .

[3]  Satyandra K. Gupta,et al.  Towards the development of a virtual environment-based training system for mechanical assembly operations , 2007, Virtual Reality.

[4]  Mansooreh Mollaghasemi,et al.  Usability engineering of virtual environments (VEs): identifying multiple criteria that drive effective VE system design , 2003, Int. J. Hum. Comput. Stud..

[5]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[6]  Michael D. McNeese,et al.  Assessment of User Affective and Belief States for Interface Adaptation: Application to an Air Force Pilot Task , 2002, User Modeling and User-Adapted Interaction.

[7]  Fabrizia Mantovani,et al.  12 VR Learning: Potential and Challenges for the Use of 3D Environments in Education and Training , 2003 .

[8]  Jim X. Chen,et al.  A Model for Understanding How Virtual Reality Aids Complex Conceptual Learning , 1999, Presence: Teleoperators & Virtual Environments.

[9]  H. Rose.  Assessing Learning in VR: Towards Developing a Paradigm. Virtual Reality Roving Vehicles (VRRV) Project. , 1995 .

[10]  E. Salas,et al.  Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. , 1993 .

[11]  William Winn,et al.  Learning in Artificial Environments: Embodiment, Embeddedness and Dynamic Adaptation , 2003 .

[12]  Bruce H. Thomas,et al.  Emerging technologies of augmented reality - interfaces and design , 2006 .

[13]  Spiro Kiousis,et al.  Interactivity: a concept explication , 2002, New Media Soc..

[14]  Fred D. Davis.  Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology , 1989, MIS Q..

[15]  Maureen Tam,et al.  Constructivism, Instructional Design, and Technology: Implications for Transforming Distance Learning , 2000, J. Educ. Technol. Soc..

[16]  R. Moreno,et al.  Does the modality principle hold for different media? A test of the method-affects-learning hypothesis , 2006, J. Comput. Assist. Learn..

[17]  Donald E. Parker,et al.  User Experience Modeling and Enhancement for Virtual Environments That Employ Wide-Field Displays , 2007, HCI.

[18]  Matthew Turk,et al.  Perceptual user interfaces , 2000 .

[19]  Jon Pearce,et al.  An investigation of interactivity and flow: student behaviour during online instruction , 2004 .