Measuring the Learning Effectiveness of Serious Gaming for Training of Complex Manufacturing Tasks

Background. Training new workers on complex manufacturing tasks has long been a challenge for high value manufacturing companies. Equipment downtime, costly instructors, and dangerous working environments are some of the impediments of hands-on training. To overcome these hurdles, a traditional manufacturing paper manual was transformed into a serious game through capturing and embedding expert knowledge. Aim. This article investigates the learning effectiveness of learning via a serious game (Training Game) compared with the tradition learning method (Paper Manual) through a user study. Method. Twenty employees took part in a randomised controlled trial. They were assigned to one of two conditions: Training Game (experimental condition), or Paper Manual (control condition). Participants spent a maximum of 30 minutes to study manufacturing instructions before completing two tests to evaluate the amount of learning achieved. Results. The results show that the Training Game was more effective for learning procedural knowledge than the Paper Manual. Regarding factual knowledge, no significant difference was identified between the two conditions. In terms of motivation, increased engagement levels were reported in the Training Game condition. Conclusions. This user study shows evidence that the serious TG being evaluated is an effective method for training procedural knowledge in a complex manufacturing scenario.

[1]  Lakhmi C. Jain,et al.  Advanced Computational Intelligence Paradigms in Healthcare 6. Virtual Reality in Psychotherapy, Rehabilitation, and Assessment , 2011 .

[2]  Huiru Zheng,et al.  Virtual Reality and Serious Games in Healthcare , 2011 .

[3]  Lorenzo Seidenari,et al.  Emergency medicine training with gesture driven interactive 3D simulations , 2012, UXeLATE '12.

[4]  Julian H. Kang,et al.  Merit of Computer Game in Tacit Knowledge Acquisition and Retention , 2011 .

[5]  Margaret E. Gredler,et al.  Games and Simulations and Their Relationships to Learning , 2013 .

[6]  Andrea F. Abate,et al.  A haptic-based approach to virtual training for aerospace industry , 2009, J. Vis. Lang. Comput..

[7]  James E. Driskell,et al.  Games, Motivation, and Learning: A Research and Practice Model , 2002 .

[8]  Mert Bal.  Virtual Manufacturing Laboratory Experiences for Distance Learning Courses in Engineering Technology , 2012 .

[9]  Katherine A Sward,et al.  Use of a Web-based game to teach pediatric content to medical students. , 2008, Ambulatory pediatrics : the official journal of the Ambulatory Pediatric Association.

[10]  H. O'Neil,et al.  Classification of learning outcomes: evidence from the computer games literature , 2005 .

[11]  Mar González-Franco,et al.  Empowering patients to perform physical therapy at home , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[12]  Thomas Connolly,et al.  Games-Based Learning Advancements for Multi-Sensory Human Computer Interfaces: Techniques and Effective Practices , 2009 .

[13]  Iván Martínez-Ortiz,et al.  Development of Game-Like Simulations for Procedural Knowledge in Healthcare Education , 2014, IEEE Transactions on Learning Technologies.

[14]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[15]  Riccardo Berta,et al.  Assessment in and of Serious Games: An Overview , 2013, Adv. Hum. Comput. Interact..

[16]  J. Lanier,et al.  Model of Illusions and Virtual Reality , 2017, Front. Psychol..

[17]  Luca Chittaro,et al.  Assessing Knowledge Retention of an Immersive Serious Game vs. a Traditional Education Method in Aviation Safety , 2015, IEEE Transactions on Visualization and Computer Graphics.

[18]  S. K. Ong,et al.  Virtual reality simulations and animations in a web-based interactive manufacturing engineering module , 2004, Comput. Educ..

[19]  Glyn Lawson,et al.  Empirical evidence, evaluation criteria and challenges for the effectiveness of virtual and mixed reality tools for training operators of car service maintenance , 2015, Comput. Ind..

[20]  P. Wouters,et al.  A meta-analysis of the cognitive and motivational effects of serious games , 2013 .

[21]  Anissa All,et al.  Measuring Effectiveness in Digital Game-Based Learning: A Methodological Review , 2014, Int. J. Serious Games.

[22]  Jean Écalle,et al.  Serious games as new educational tools: how effective are they? A meta-analysis of recent studies , 2013, J. Comput. Assist. Learn..

[23]  David B. Kaber,et al.  The need for worker training in advanced manufacturing technology (AMT) environments: A white paper , 1999 .

[24]  L. Buchanan,et al.  Blending Bloom ’ s Taxonomy and Serious Game Design , 2011 .

[25]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[26]  Hiran Ekanayake,et al.  Game interaction state graphs for evaluation of user engagement in explorative and experience-based training games , 2010, 2010 International Conference on Advances in ICT for Emerging Regions (ICTer).

[27]  Tabitha C. Peck,et al.  A threat to a virtual hand elicits motor cortex activation , 2014, Experimental Brain Research.

[28]  Traci Sitzmann.  A META-ANALYTIC EXAMINATION OF THE INSTRUCTIONAL EFFECTIVENESS OF COMPUTER-BASED SIMULATION GAMES , 2011 .

[29]  James F. Knight,et al.  Serious gaming technology in major incident triage training: a pragmatic controlled trial. , 2010, Resuscitation.

[30]  Ashutosh Tiwari,et al.  Immersive Mixed Reality for Manufacturing Training , 2017, Front. Robot. AI.

[31]  B. Kapralos,et al.  Using Serious Games and Virtual Simulation for Training in the Fire Service: A Review , 2015 .

[32]  Harvey Maylor,et al.  GAME PLAYING AND OPERATIONS MANAGEMENT EDUCATION , 2007 .

[33]  Herre van Oostendorp,et al.  Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective , 2009 .

[34]  D. Krathwohl.  A Revision of Bloom's Taxonomy: An Overview , 2002 .

[35]  Anissa All,et al.  Towards a conceptual framework for assessing the effectiveness of digital game-based learning , 2015, Comput. Educ..

[36]  S. Cole,et al.  Improvement in cancer-related knowledge following use of a psychoeducational video game for adolescents and young adults with cancer. , 2007, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[37]  Marina Papastergiou,et al.  Digital Game-Based Learning in high school Computer Science education: Impact on educational effectiveness and student motivation , 2009, Comput. Educ..

[38]  Yizhe Chang,et al.  Virtual Mechanical Assembly Training Based on a 3D Game Engine , 2015 .

[39]  P. Chandler,et al.  Assimilating complex information , 2002 .

[40]  Roberto Canas,et al.  Game-based trench safety education: development and lessons learned , 2011, J. Inf. Technol. Constr..

[41]  Stefan Greuter,et al.  Designing a game for occupational health and safety in the construction industry , 2012, IE '12.

[42]  Danielle S. McNamara,et al.  Short and Long Term Benefits of Enjoyment and Learning within a Serious Game , 2011, AIED.

[43]  Konstantin Mitgutsch,et al.  Purposeful by design?: a serious game design assessment framework , 2012, FDG.

[44]  Tabitha C. Peck,et al.  Putting yourself in the skin of a black avatar reduces implicit racial bias , 2013, Consciousness and Cognition.

[45]  Shu-Huang Sun,et al.  Development of virtual training platform of injection molding machine based on VR technology , 2012 .

[46]  Benjamin S. Bloom,et al.  A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives , 2000 .