Challenges in the Development and Evaluation of Immersive Digital Educational Games

In this paper, we describe a conceptual framework and address the related issues and solutions in the identification of three major challenges for the development and evaluation of Immersive Digital Educational Games (IDEGs). These challenges are (i) advancing adaptive educational technologies to shape learning experience, ensuring the individualization of learning experiences, adaptation to personal aims, needs, abilities and prerequisites; (ii) providing technological approaches to reduce the development costs for IDEGs; by enabling the creation of entirely different stories and games for a variety of different learning domains, each based, more or less, on the same pool of story units; patterns and structures; (iii) developing robust evaluation methodologies for IDEGs by the extension of ISO 9241 to include user satisfaction, motivation and learning progress and other User Experience (UX) attributes. While our research and development is by no means concluded, we believe that we have arrived at a stage where conclusions may be drawn, which will be of considerable use to other researchers in this domain.

[1]  Andreas Beu,et al.  Engineering Joy , 2001, IEEE Softw..

[2]  Fabio Paternò,et al.  Human-Computer Interaction - INTERACT 2005 , 2005, Lecture Notes in Computer Science.

[3]  A. Conchon,et al.  Applications and services , 2001 .

[4]  Hermann A. Maurer,et al.  TRIANGLE: A Multi-Media test-bed for examining incidental learning, motivation and the Tamagotchi-Effect within a Game-Show like Computer Based Learning Module , 2001 .

[5]  K. Scherer,et al.  Psychophysiological responses to appraisal dimensions in a computer game , 2004 .

[6]  Andreas Holzinger,et al.  A Study about Motivation in Computer Aided Mathematics Instruction with Mathematica 3.0 , 1997 .

[7]  F. Paas,et al.  Interactivity in Video-based Models , 2007 .

[8]  Miguel Nussbaum,et al.  Playability in Action Videogames: A Qualitative Design Model , 2002, Hum. Comput. Interact..

[9]  Dietrich Albert,et al.  Knowledge Spaces: Theories, Empirical Research, and Applications , 1998 .

[10]  Heather Desurvire,et al.  Using heuristics to evaluate the playability of games , 2004, CHI EA '04.

[11]  Regan L. Mandryk,et al.  A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies , 2007, Int. J. Hum. Comput. Stud..

[12]  Stefanie Harbich,et al.  Rater Bias: The Influence of Hedonic Quality on Usability Questionnaires , 2005, INTERACT.

[13]  Velda McCune,et al.  The Conceptual Bases of Study Strategy Inventories , 2004 .

[14]  Stephanie Rosenbaum,et al.  A toolkit for strategic usability: results from workshops, panels, and surveys , 2000, CHI.

[15]  Andreas Holzinger,et al.  Rapid prototyping for a virtual medical campus interface , 2004, IEEE Software.

[16]  Paul van Schaik,et al.  Modelling user experience with web sites: Usability, hedonic value, beauty and goodness , 2008, Interact. Comput..

[17]  Muriel Garreta Domingo,et al.  Ten emotion heuristics: guidelines for assessing the user's affective dimension easily and cost-effectively , 2007, BCS HCI.

[18]  Scott P. Robertson,et al.  The development of an interactive game-based tool for learning surgical management algorithms via computer. , 2002, American journal of surgery.

[19]  Martin Ebner,et al.  Useful oblivion versus Information Overload in e-learning examples in the context of Wiki systems , 2008, ITI 2008 - 30th International Conference on Information Technology Interfaces.

[20]  M. David Merrill,et al.  First principles of instruction , 2012 .

[21]  Martin Oliver,et al.  How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? , 2006, Comput. Educ..

[22]  Manfred Tscheligi,et al.  Evaluating user experiences in games , 2008, CHI Extended Abstracts.

[23]  Carsten Röcker,et al.  Exploring the Usability of Video Game Heuristics for Pervasive Game Development in Smart Home Environments , 2006 .

[24]  Michael Burmester,et al.  Hedonic and ergonomic quality aspects determine a software's appeal , 2000, CHI.

[25]  Andreas Holzinger,et al.  Multi Media e-Learning Software TRIANGLE Case-Study: Experimental Results and Lessons Learned , 2006 .

[26]  Josef Fink,et al.  Enhancing Universal Access - EEG Based Learnability Assessment , 2007, HCI.

[27]  M. Csíkszentmihályi.  Finding Flow: The Psychology of Engagement with Everyday Life , 1997 .

[28]  Andreas Holzinger,et al.  Dynamic Media in Computer Science Education; Content Complexity and Learning Performance: Is Less More? , 2008, J. Educ. Technol. Soc..

[29]  Martin Ebner,et al.  Successful implementation of user-centered game based learning in higher education: An example from civil engineering , 2007, Comput. Educ..

[30]  Muriel Garreta-Domingo,et al.  Ten emotion heuristics: guidelines for assessing the user's affective dimension easily and cost-effectively , 2007 .

[31]  Manfred Tscheligi,et al.  Methods for evaluating games: how to measure usability and user experience in games? , 2007, ACE '07.

[32]  Andreas Holzinger,et al.  Usability engineering methods for software developers , 2005, CACM.

[33]  Ben Shneiderman,et al.  Relate-Create-Donate: a teaching/learning philosophy for the cyber-generation , 1998, Comput. Educ..

[34]  Judith Klein-Seetharaman,et al.  Towards Biomedical Problem Solving in a Game Environment , 2003, International Conference on Computational Science.

[35]  M. Lepper,et al.  Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. , 1996 .

[36]  Judy Robertson,et al.  Computer game design: Opportunities for successful learning , 2008, Comput. Educ..

[37]  A. Bandura,et al.  Longitudinal Analysis of the Role of Perceived Self-Efficacy for Self-Regulated Learning in Academic Continuance and Achievement. , 2008 .

[38]  James E. Driskell,et al.  Games, Motivation, and Learning: A Research and Practice Model , 2002 .

[39]  Dietrich Albert,et al.  The ELEKTRA Ontology Model: A Learner-Centered Approach to Resource Description , 2007, ICWL.

[40]  H WidemanHerbert,et al.  Unpacking the potential of educational gaming , 2007 .

[41]  Menno D. T. de Jong,et al.  Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison , 2004, Interact. Comput..

[42]  Elina M. I. Koivisto,et al.  Playability heuristics for mobile multi-player games , 2007, DIMEA.

[43]  Menno D. T. de Jong,et al.  Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online library catalogue , 2003, Behav. Inf. Technol..

[44]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[45]  Jack Dongarra,et al.  Computational Science — ICCS 2003 , 2003, Lecture Notes in Computer Science.

[46]  B. Lockee,et al.  Measuring Success: Evaluation Strategies for Distance Education , 2002 .

[47]  M. Boekaerts.  SELF-REGULATED LEARNING: A NEW CONCEPT EMBRACED BY RESEARCHERS, POLICY MAKERS, EDUCATORS, TEACHERS, AND STUDENTS , 1997 .

[48]  Christine Brown,et al.  Unpacking the potential of educational gaming: A new tool for gaming research , 2007 .

[49]  Martin Schrepp,et al.  The influence of hedonic quality on the attractiveness of user interfaces of business management software , 2006, Interact. Comput..