Adaptive Experience Engine for Serious Games

Designing games that support knowledge and skill acquisition has become a promising frontier of education techniques, since games are able to capture the user concentration for long periods and can present users with realistic and compelling challenges. In this scenario, there is a need for scientific and engineering methods to build games not only as more realistic simulations of the physical world but as means to provide effective learning experiences. Abstracting state of the art serious games' (SGs) features, we propose a new design methodology for the sand box serious games (SBSGs) class, decoupling content from the delivery strategy during the gameplay. This methodology aims at making design more efficient and standardized in order to meet the growing demand for interactive learning. The methodology consists in modeling an SBSG as a hierarchy of tasks (e.g., missions) and specifies the requirements for a runtime scheduling policy that maximizes learning objectives in a full entertainment context. The policy is learned by an experience engine (EE) based on computational intelligence. In this approach, the domain-expert author focuses on the creation and semantic annotation of tasks. Tasks are put in a repository and can then be exploited by game designers who define the expected learning curve and other requirements about education and entertainment for the game. The task sequencing that aims at matching such specifications with the real user profile is then presented to the EE. The EE can operate also in absence of the specification of the learning curve, continuously adapting the game flow without aiming at the achievement of target knowledge levels predefined by the author. We have implemented an EE module based on genetic computation and reinforcement learning (RL) atop of a state-of-the-art game engine. Test results show that the EE is able to define in real-time missions that meet the requirements expressed by the author.

[1]  Andrew G. Barto,et al.  Learning to Act Using Real-Time Dynamic Programming , 1995, Artif. Intell..

[2]  Brian Magerko Adaptation in Digital Games , 2008, Computer.

[3]  Jane Willis,et al.  A Framework for Task-Based Learning , 1996 .

[4]  J. P. Gee,et al.  What Would a State of the Art Instructional Video Game Look like , 2005 .

[5]  Geoff F. Kaufman,et al.  Understanding Media Enjoyment: The Role of Transportation Into Narrative Worlds , 2004 .

[6]  Alessandro De Gloria,et al.  A task annotation model for Sandbox Serious Games , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[7]  Jenova Chen,et al.  Flow in games (and everything else) , 2007, CACM.

[8]  Gary B. Wills,et al.  Is Less Actually More? The Usefulness of Educational Mini-games , 2007, Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007).

[9]  Lisa Gualtieri,et al.  Serious games for serious topics , 2008, ELERN.

[10]  Ian D. Watson,et al.  Using reinforcement learning for city site selection in the turn-based strategy game Civilization IV , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[11]  Etienne Wenger,et al.  Artificial Intelligence and Tutoring Systems: Computational and Cognitive Approaches to the Communication of Knowledge , 1987 .

[12]  Martin Damsbo,et al.  An Evolutionary Algorithm for Welding Task Sequence Ordering , 1998, AISC.

[13]  Jonathan P. Rowe,et al.  Motivation in Narrative-Centered Learning Environments , 2007 .

[14]  Chris Anderson The long tail : how endless choice is creating unlimited demand , 2006 .

[15]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[16]  Alessandro De Gloria,et al.  Exploring gaming mechanisms to enhance knowledge acquisition in virtual worlds , 2008, DIMEA.

[17]  David Murphy,et al.  JDoc: A Serious Game for Medical Learning , 2008, First International Conference on Advances in Computer-Human Interaction.

[18]  Marcus Gallagher,et al.  Reinforcement Learning in First Person Shooter Games , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[19]  Jürgen Schmidhuber,et al.  Learning Team Strategies: Soccer Case Studies , 1998, Machine Learning.

[20]  Rudy Darken,et al.  The Delta3D Open Source Game Engine , 2005, IEEE Computer Graphics and Applications.

[21]  Gerald Tesauro,et al.  Temporal Difference Learning and TD-Gammon , 1995, J. Int. Comput. Games Assoc..

[22]  Andrew Stern,et al.  Believable Agents and Intelligent Story Adaptation for Interactive Storytelling , 2006, TIDSE.

[23]  Mark O. Riedl,et al.  Dynamic Experience Management in Virtual Worlds for Entertainment , Education , and Training , 2008 .

[24]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[25]  Mahesan Niranjan,et al.  On-line Q-learning using connectionist systems , 1994 .

[26]  Justin A. Boyan,et al.  Least-Squares Temporal Difference Learning , 1999, ICML.

[27]  Shimon Whiteson,et al.  Evolutionary Function Approximation for Reinforcement Learning , 2006, J. Mach. Learn. Res..

[28]  Alessandro De Gloria,et al.  Enhancing the educational value of video games , 2009, CIE.

[29]  Marc J. V. Ponsen,et al.  Improving Adaptive Game Ai with Evolutionary Learning , 2004 .

[30]  D. Kolb,et al.  Learning Styles and Learning Spaces: Enhancing Experiential Learning in Higher Education , 2005 .

[31]  T. Ganchev,et al.  PlayMancer: A Serious Gaming 3D Environment , 2008, 2008 International Conference on Automated Solutions for Cross Media Content and Multi-Channel Distribution.

[32]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[33]  Mel Slater,et al.  Visual Realism Enhances Realistic Response in an Immersive Virtual Environment , 2009, IEEE Computer Graphics and Applications.

[34]  Rod Ellis,et al.  Task-based Language Learning and Teaching , 2003 .

[35]  Francisco S. Melo,et al.  Q -Learning with Linear Function Approximation , 2007, COLT.

[36]  Hector Muñoz-Avila,et al.  RETALIATE: Learning Winning Policies in First-Person Shooter Games , 2007, AAAI.

[37]  Stacy Marsella,et al.  THESPIAN: An Architecture for Interactive Pedagogical Drama , 2005, AIED.

[38]  Brian S. Stensrud,et al.  Bringing the Schoolhouse Inside the Box - A Tool for Engaging, Individualized Training , 2006 .

[39]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[40]  Sonny Chan,et al.  OceanQuest: A University-Based Serious Game Project , 2005, DiGRA Conference.

[41]  Darryl Charles,et al.  Toward an understanding of flow in video games , 2008, CIE.

[42]  Richard S. Sutton,et al.  Dyna, an integrated architecture for learning, planning, and reacting , 1990, SGAR.

[43]  Valerie J. Shute,et al.  Intelligent Tutoring Systems: Past, Present, and Future. , 1994 .

[44]  K. Squire Open-Ended Video Games: A Model for Developing Learning for the Interactive Age , 2007 .

[45]  Elaine M. Raybourn,et al.  Design and evaluation challenges of serious games , 2005, CHI Extended Abstracts.

[46]  W. Lewis Johnson Lessons learned from games for education , 2005, SIGGRAPH '05.

[47]  R. Clayton,et al.  Epicardial ECG Mapping of Human Ventricular Fibrillation , 2006 .

[48]  Sue Blackman,et al.  Serious games...and less! , 2005, COMG.

[49]  Manfred Tscheligi,et al.  Methods for evaluating games: how to measure usability and user experience in games? , 2007, ACE '07.

[50]  Clark N Quinn,et al.  Engaging Learning: Designing e-Learning Simulation Games , 2005 .

[51]  A Ehrenfeucht,et al.  Organization of memory. , 1973, Proceedings of the National Academy of Sciences of the United States of America.

[52]  Andrew Moshirnia,et al.  The Educational Potential of Modified Video Games , 2007 .

[53]  Richard S. Sutton,et al.  Generalization in ReinforcementLearning : Successful Examples UsingSparse Coarse , 1996 .

[54]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[55]  Stacy Marsella,et al.  Thespian: Modeling Socially Normative Behavior in a Decision-Theoretic Framework , 2006, IVA.

[56]  James Paul Gee,et al.  What video games have to teach us about learning and literacy , 2007, CIE.

[57]  W. Higinbotham The Federation of American Scientists , 1948 .

[58]  L. S. Vygotsusuky Play and its role in the mental development of the child , 1976 .

[59]  Henry Kelly,et al.  How to build serious games , 2007, CACM.

[60]  Melissa A. Federoff,et al.  HEURISTICS AND USABILITY GUIDELINES FOR THE CREATION AND EVALUATION OF FUN IN VIDEO GAMES , 2002 .

[61]  Perry McDowell,et al.  Game AI in Delta3D , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[62]  Baltasar Fernández-Manjón,et al.  A Game-Based Adaptive Unit of Learning with IMS Learning Design and , 2007, EC-TEL.

[63]  Sanghamitra Bandyopadhyay,et al.  Genetic operators for combinatorial optimization in TSP and microarray gene ordering , 2007, Applied Intelligence.

[64]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[65]  Arthur A. Raney Expanding Disposition Theory: Reconsidering Character Liking, Moral Evaluations, and Enjoyment , 2004 .

[66]  Marina Krcmar,et al.  Conceptualizing Media Enjoyment as Attitude: Implications for Mass Media Effects Research , 2004 .

[67]  Frank L. Greitzer,et al.  Cognitive science implications for enhancing training effectiveness in a serious gaming context , 2007, JERC.

[68]  H. W. Hodgins The Future of Learning Objects , 2002 .

[69]  Robert Michael Young,et al.  From linear story generation to branching story graphs , 2005, IEEE Computer Graphics and Applications.

[70]  A. Chris,et al.  Long Tail: How Endless Choice is Creating Unlimited Demand, London: Random House. , 1996 .

[71]  Jane R. Willis,et al.  Challenge and Change in Language Teaching , 1996 .

[72]  Thomas W. Malone,et al.  What Makes Things Fun to Learn? A Study of Intrinsically Motivating Computer Games. , 1981 .

[73]  Vincent P. Wade,et al.  Adaptive Educational Games: Providing Non-invasive Personalised Learning Experiences , 2008, 2008 Second IEEE International Conference on Digital Game and Intelligent Toy Enhanced Learning.

[74]  Peta Wyeth,et al.  GameFlow: a model for evaluating player enjoyment in games , 2005, CIE.

[75]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .