Assessing the Reach and Impact of Game-Based Learning Approaches to Cultural Competency and Behavioural Change

As digital games continue to be explored as solutions to educational and behavioural challenges, the need for evaluation methodologies which support both the unique nature of the format and the need for comparison with other approaches continues to increase. In this workshop paper, a range of challenges are described related specifically to the case of cultural learning using digital games, in terms of how it may best be assessed, understood, and sustained through an iterative process supported by research. An evaluation framework is proposed, identifying metrics for reach and impact and their associated challenges, as well as presenting ethical considerations and the means to utilize evaluation outcomes within an iterative cycle, and to provide feedback to learners. Presenting as a case study a serious game from the Mobile Assistance for Social Inclusion and Empowerment of Immigrants with Persuasive Learning Technologies and Social Networks (MASELTOV) project, the use of the framework in the context of an integrative project is discussed, with emphasis on the need to view game-based learning as a blended component of the cultural learning process, rather than a standalone solution. The particular case of mobile gaming is also considered within this case study, providing a platform by which to deliver and update content in response to evaluation outcomes. Discussion reflects upon the general challenges related to the assessment of cultural learning, and behavioural change in more general terms, suggesting future work should address the need to provide sustainable, research-driven platforms for game-based learning content.

[1]  P. Dolan,et al.  Influencing behaviour: The mindspace way , 2012 .

[2]  C. Cooper.  Theories of Group Processes , 1976 .

[3]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[4]  Tim Marsh,et al.  Experience in Serious Games: Between Positive and Serious Experience , 2012, SGDA.

[5]  Roger Altizer,et al.  The intersection of video games and patient empowerment: case study of a real world application , 2013, IE.

[6]  Katherine Isbister,et al.  Designing games for learning: insights from conversations with designers , 2010, CHI.

[7]  Alexandra Poulovassilis,et al.  Developing an Evaluation Methodology for Immersive Learning Experiences in a Virtual World , 2009, 2009 Conference in Games and Virtual Worlds for Serious Applications.

[8]  Taiki Takahashi,et al.  Discounting delayed and probabilistic monetary gains and losses by smokers of cigarettes , 2005, Psychopharmacology.

[9]  Paula Escudeiro,et al.  Evaluation of Serious Games in Mobile Platforms with QEF: QEF (Quantitative Evaluation Framework) , 2012, 2012 IEEE Seventh International Conference on Wireless, Mobile and Ubiquitous Technology in Education.

[10]  David Stevens,et al.  Developing Intercultural Competence in Practice , 2001 .

[11]  Chee Siang Ang,et al.  E-learning as Computer Games: Designing Immersive and Experiential Learning , 2004, PCM.

[12]  Brian Webb,et al.  Stand up and take your place: identifying narrative elements in narrative adventure and role-play games , 2005, CIE.

[13]  Mark A. Elliott,et al.  ADOLESCENT ROAD USER BEHAVIOUR: A SURVEY OF 11-16 YEAR OLDS , 2003 .

[14]  W. Lewis Johnson.  Developing intercultural competence through videogames , 2009, IWIC '09.

[15]  Zhen Jiao,et al.  Tutoring Strategy Study on Game-Based Experiential Learning in Vocational School , 2009, 2009 First International Workshop on Education Technology and Computer Science.

[16]  Paul A. Cairns,et al.  Video games as research instruments , 2010, CHI EA '10.

[17]  Emmanuel G. Blanchard,et al.  EAGLE: An Intelligent Tutoring System to Support Experiential Learning Through Video Games , 2009, AIED.

[18]  M. Carrillo-de-la-Peña,et al.  The components of impulsiveness: A comparison of the I.7 impulsiveness questionnaire and the Barratt impulsiveness scale , 1991 .

[19]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[20]  Sobah Abbas Petersen,et al.  Challenges and Opportunities in Evaluating Learning in Serious Games: A Look at Behavioural Aspects , 2012, SGDA.

[21]  Claudine Kirsch.  Developing intercultural competence , 2008 .

[22]  Charles A. Holt,et al.  Risk Aversion and Incentive Effects , 2002 .

[23]  Ronald Baecker,et al.  A framework for the online evaluation of serious games , 2010, Future Play.

[24]  Alexander Verbraeck,et al.  The participatory design of a simulation training game , 2012, Proceedings Title: Proceedings of the 2012 Winter Simulation Conference (WSC).

[25]  Richard M. Crowder,et al.  Validation of Serious Games Attributes Using the Technology Acceptance Model , 2010, 2010 Second International Conference on Games and Virtual Worlds for Serious Applications.

[26]  Martin Ruskov,et al.  Dealing with Threshold Concepts in Serious Games for Competence Development , 2012, SGDA.

[27]  Riccardo Berta,et al.  Assessment in and of Serious Games: An Overview , 2013, Adv. Hum. Comput. Interact..

[28]  Dietrich Albert,et al.  Tracking the UFO's Paths: Using Eye-Tracking for the Evaluation of Serious Games , 2011, HCI.

[29]  Elaine M. Raybourn,et al.  Design and evaluation challenges of serious games , 2005, CHI Extended Abstracts.