Assessing Performance Competence in Training Games

In-process assessment of trainee learners in game-based simulators is a challenging activity. This typically involves human instructor time and cost, and does not scale to the one tutor per learner vision of computer-based learning. Moreover, evaluation from a human instructor is often subjective and comparisons between learners are not accurate. Therefore, in this paper, we propose an automated, formula-driven quantitative evaluation method for assessing performance competence in serious training games. Our proposed method has been empirically validated in a game-based driving simulator using 7 subjects and 13 sessions, and accuracy up to 90.25% has been achieved when compared to an existing qualitative method. We believe that by incorporating quantitative evaluation methods like these future training games could be enriched with more meaningful feedback and adaptive game-play so as to better monitor and support player motivation, engagement and learning performance.

[1]  Lakshmi Goel,et al.  Situated Learning: Conceptualization and Measurement , 2010 .

[2]  G. Gibbs,et al.  Preparing to teach : an introduction to effective teaching in higher education , 1992 .

[3]  James Paul Gee,et al.  Deep Learning Properties of Good Digital Games: How Far Can They Go? , 2009 .

[4]  Henrik Engström,et al.  Games for traffic education: An experimental study of a game-based driving simulator , 2010 .

[5]  Peggy A. Ertmer,et al.  Behaviorism, Cognitivism, Constructivism: Comparing Critical Features From an Instructional Design Perspective , 2008 .

[6]  Cynthia Breazeal,et al.  Affective Learning — A Manifesto , 2004 .

[7]  Coniferous softwood.  GENERAL TERMS , 2003 .

[8]  Eva Hudlicka,et al.  Affective game engines: motivation and requirements , 2009, FDG.

[9]  Jessica D. Bayliss,et al.  The Classroom is a Game ? , 2009 .

[10]  Johannes Wagner,et al.  Emote toWin: Affective Interactions with a Computer Game Agent , 2004, GI Jahrestagung.

[11]  Richard N. Van Eck.  Digital Game-Based Learning: It's Not Just the Digital Natives Who Are Restless. , 2006 .

[12]  P. Backlund,et al.  Designing for Self-Efficacy in a Game Based Simulator: An Experimental Study and Its Implications for Serious Games Design , 2008, 2008 International Conference Visualisation.

[13]  Angela Brennecke,et al.  A general framework for digital game-based training systems , 2009 .

[14]  Per Backlund,et al.  Sidh - a Game Based Firefighter Training Simulation , 2007, 2007 11th International Conference Information Visualization (IV '07).

[15]  Hiran Ekanayake,et al.  Game interaction state graphs for evaluation of user engagement in explorative and experience-based training games , 2010, 2010 International Conference on Advances in ICT for Emerging Regions (ICTer).

[16]  Brian McDonald,et al.  Intelligent Biofeedback using an Immersive Competitive Environment , 2001 .

[17]  J. Harackiewicz,et al.  Motivating the Academically Unmotivated: A Critical Issue for the 21st Century , 2000 .

[18]  George D. Kuh,et al.  Student Engagement and Student Learning: Testing the Linkages* , 2006 .

[19]  A. Elliot,et al.  A HIERARCHICAL MODEL OF APPROACH AND AVOIDANCE ACHIEVEMENT MOTIVATION , 1997 .

[20]  Frank C. Leeming,et al.  Deep Learning and Emotion in Serious Games , 2009 .

[21]  Gavriel Salomon,et al.  T RANSFER OF LEARNING , 1992 .

[22]  Alan J. Dix,et al.  Affective Videogames and Modes of Affective Gaming: Assist Me, Challenge Me, Emote Me (ACE) , 2005, DiGRA Conference.

[23]  C. Bray.  Transfer of learning. , 1928 .

[24]  Mark Claypool,et al.  Relating cognitive models of computer games to user evaluations of entertainment , 2009, FDG.

[25]  T. Husén,et al.  The International Encyclopedia of Education , 1994 .