Evaluating the educational effectiveness of simulation games: A value generation model

This article investigates the relationships between various types of educational value generated by the Markstrat simulation game. Considering several theoretical models of experiential learning and the research framework proposed by previous studies, an educational value generation model is developed and validated, using primary data collected from 305 UK-based students. Four types of educational value are identified: experience generation, conceptual understanding, skills development, and affective evaluation. The application of structural equation modelling indicates several significant relationships: experience generation has a strong impact on conceptual understanding, and both of them have medium to high direct impacts on skills development. On the other hand, the participants' perception regarding the professional skills developed during the simulation game determines their affective evaluation of the Markstrat exercise. The model presented in this study is generalizable to other simulation games, and to other academic disciplines that implement the same experiential learning approach.

[1]  Gráinne Conole,et al.  The influence of pervasive and integrative tools on learners' experiences and expectations of study , 2010 .

[2]  Louise Sauvé,et al.  Distinguishing between Games and Simulations: A Systematic Review , 2007, J. Educ. Technol. Soc..

[3]  Paul Maharg,et al.  Introduction: Digital Games and Learning , 2011 .

[4]  Richard P. Bagozzi,et al.  Assessing Construct Validity in Organizational Research , 1991 .

[5]  Craig A. Lindley.  Trans-Reality Gaming 1 , 2004 .

[6]  David Gray,et al.  The Application of Self-Directed Learning in a Marketing Strategy Capstone Course , 2011 .

[7]  Henry Jenkins.  Getting into the Game , 2005 .

[8]  Howard S. Adelman,et al.  Intervention Theory and Evaluating Efficacy , 1986 .

[9]  Robert W. Cook,et al.  The Pedagogical Efficacy of a Sales Management Simulation , 2006 .

[10]  June Lee,et al.  Effectiveness of Computer-Based Instructional Simulation: A Meta Analysis. , 1999 .

[11]  D. Goldenberg,et al.  The effect of classroom simulation on nursing students' self-efficacy related to health teaching. , 2005, The Journal of nursing education.

[12]  Federico Pasin,et al.  The impact of a simulation game on operations management education , 2011, Comput. Educ..

[13]  Riccardo Berta,et al.  Assessment in and of Serious Games: An Overview , 2013, Adv. Hum. Comput. Interact..

[14]  Lawrence M. Lamont,et al.  Enhancing Student and Team Learning with Interactive Marketing Simulations , 2001 .

[15]  R. Bagozzi,et al.  On the evaluation of structural equation models , 1988 .

[16]  Rex B. Kline,et al.  Principles and Practice of Structural Equation Modeling , 1998 .

[17]  Lynn Vos,et al.  Student experiences of the use of a marketing simulation game , 2008 .

[18]  David Tonks,et al.  Using Marketing Simulations for Teaching and Learning , 2002 .

[19]  Sara de Freitas,et al.  Using games and simulations for supporting learning , 2006 .

[20]  Jane Marie Cioffi,et al.  A pilot study to investigate the effect of a simulation strategy on the clinical decision making of midwifery students. , 2005, The Journal of nursing education.

[21]  Traci Sitzmann.  A META-ANALYTIC EXAMINATION OF THE INSTRUCTIONAL EFFECTIVENESS OF COMPUTER-BASED SIMULATION GAMES , 2011 .

[22]  Debra A. Laverie.  In-Class Active Cooperative Learning: A Way to Build Knowledge and Skills in Marketing Courses , 2006 .

[23]  David Tonks,et al.  A framework for understanding learning from management simulations , 1997, J. Comput. Assist. Learn..

[24]  C. Estes.  Promoting Student-Centered Learning in Experiential Education , 2004 .

[25]  Clark Aldrich,et al.  Learning by Doing: A Comprehensive Guide to Simulations, Computer Games, and Pedagogy in e-Learning and Other Educational Experiences , 2005 .

[26]  Jessica L. Wildman,et al.  Using Simulation-Based Training to Enhance Management Education , 2009 .

[27]  James E. Driskell,et al.  Games, Motivation, and Learning: A Research and Practice Model , 2002 .

[28]  Trish Andrews,et al.  Incorporating authentic learning experiences within a university course , 2004 .

[29]  G. Hatano,et al.  TWO COURSES OF EXPERTISE , 1984 .

[30]  Geert Vissers,et al.  A Simple Classification Model for Debriefing Simulation Games , 2004 .

[31]  David C. Sharp,et al.  More Than Fun and Games: Reconsidering the Virtues of Strategic Management Simulations , 2005 .

[32]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[33]  C. J. Whitehead,et al.  An Experiential-Cognitive Methodology in the First Course in Management: Some Preliminary Results , 1975 .

[34]  C. Fornell,et al.  Evaluating structural equation models with unobservable variables and measurement error. , 1981 .

[35]  T. Fenwick.  Learning Through Experience: Troubling Orthodoxies and Intersecting Questions , 2003 .

[36]  Ernest Adams,et al.  Andrew Rollings and Ernest Adams on Game Design , 2003 .

[37]  Rex C. Mitchell.  Combining Cases and Computer Simulations in Strategic Management Courses , 2004 .

[38]  J. Wolfe,et al.  The Role of Management Games and Simulations in Education and Research , 1990 .

[39]  Min Chi,et al.  Adaptive Expertise as Acceleration of Future Learning A Case Study , 2011 .

[40]  Karl G. Jöreskog,et al.  Lisrel 8: Structural Equation Modeling With the Simplis Command Language , 1993 .

[41]  P. Anderson,et al.  Business Simulations and Cognitive Learning , 2009 .

[42]  Mary A. Swanson,et al.  Health Jeopardy: A Game to Market School Health Services , 2001, The Journal of school nursing : the official publication of the National Association of School Nurses.

[43]  J. Bransford.  How people learn , 2000 .

[44]  Arthur J. Baroody.  The development of adaptive expertise and flexibility: The integration of conceptual and procedural knowledge , 2003 .

[45]  Keith J. Topping,et al.  Individual and group learning of the Highway Code: comparing board game and traditional methods , 1998 .

[46]  C. Horne,et al.  Using simulation technology for undergraduate nursing education. , 2005, The Journal of nursing education.

[47]  Matthew Warren,et al.  Teaching Introductory Management Using A Computer Simulation Game , 2000, ICSTM.

[48]  E. Brian Peach,et al.  What are Simulations For?: Learning Objectives as a Simulation Selection Device , 2014 .

[49]  K. Lynch,et al.  Group Project Work and Student-centred Active Learning: Two different experiences , 2000 .

[50]  D. Dunn,et al.  Experiential Learning , 2019, High Impact Teaching for Sport and Exercise Psychology Educators.

[51]  Benjamin S. Bloom,et al.  Taxonomy of Educational Objectives: The Classification of Educational Goals. , 1957 .

[52]  Jerry Gosenpud.  EVALUATION OF EXPERIENTIAL LEARNING , 2001 .

[53]  Fengfeng Ke,et al.  A Qualitative Meta-Analysis of Computer Games as Learning Tools , 2009 .

[54]  John Kirriemuir,et al.  Literature Review in Games and Learning , 2004 .

[55]  Yu-Ling Lin,et al.  The values of college students in business simulation game: A means-end chain approach , 2012, Comput. Educ..

[56]  Michael John Towler,et al.  An exploration of student perception of a business simulation game , 2009 .

[57]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[58]  Katherine S. Dabbour,et al.  Applying Active Learning Methods to the Design of Library Instruction for a Freshman Seminar , 1997 .

[59]  John Washbush,et al.  A Review of Scholarship on Assessing Experiential Learning Effectiveness , 2004 .

[60]  Ann Dowker,et al.  The development of arithmetic concepts and skills: Constructing adaptive expertise , 2003 .

[61]  Sara de Freitas,et al.  The use of 'exploratory learning' for supporting immersive learning in virtual environments , 2009, Comput. Educ..

[62]  William D. Perreault,et al.  The Marketing Game , 2001 .

[63]  Larry Johnson,et al.  The 2011 Horizon Report. , 2011 .

[64]  David Jennings,et al.  Strategic Management: An Evaluation of the Use of Three Learning Methods. , 2002 .

[65]  Andrew Martin.  Adding value to simulation/games through Internet mediation: The medium and the message , 2003 .

[66]  Gary J. Summers.  Today’s Business Simulation Industry , 2004 .

[67]  Davin Pavlas,et al.  A Model of Flow and Play in Game-based Learning: The Impact of Game Characteristics, Player Traits, and Player States , 2010 .

[68]  Rolph E. Anderson,et al.  Multivariate Data Analysis with Readings , 1979 .

[69]  Terry Hyland,et al.  Experiential learning, competence and critical practice in higher education , 1994 .

[70]  David Tonks.  The processing and pedagogy of marketing simulations , 2005 .

[71]  Richard Newman,et al.  Evaluation of students' experiences of developing transferable skills and business skills using a business simulation game , 2009, 2009 39th IEEE Frontiers in Education Conference.

[72]  B. Thompson,et al.  Factor Analytic Evidence for the Construct Validity of Scores: A Historical Overview and Some Guidelines , 1996 .

[73]  Gordon Smith,et al.  Modes of learning in the use of a computer-based business simulation game , 2006, Int. J. Learn. Technol..

[74]  Matthew T. Huss,et al.  An Active-Learning Approach to Teaching the Undergraduate Psychology and Law Course , 1996 .

[75]  Szu-Yuan Sun,et al.  What influences college students to continue using business simulation games? The Taiwan experience , 2009, Comput. Educ..

[76]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[77]  FreitasSara de,et al.  The use of 'exploratory learning' for supporting immersive learning in virtual environments , 2009 .

[78]  Parshotam Dass,et al.  Improving Students’ Self-Efficacy in Strategic Management: The Relative Impact of Cases and Simulations , 2000 .

[79]  J. Cunningham.  History , 2007, The Journal of Hellenic Studies.

[80]  Hugh M. Cannon,et al.  Constructs of Simulation Evaluation , 2002 .

[81]  David W. Gerbing,et al.  An Updated Paradigm for Scale Development Incorporating Unidimensionality and Its Assessment , 1988 .

[82]  H. O'Neil,et al.  Classification of learning outcomes: evidence from the computer games literature , 2005 .

[83]  K. Holyoak.  Symbolic connectionism: Toward third-generation theories of expertise. , 1991 .

[84]  P. Wouters,et al.  A meta-analysis of the cognitive and motivational effects of serious games , 2013 .

[85]  Jeffrey Chin,et al.  Assessment in Simulation and Gaming , 2009 .

[86]  Keith A. Willoughby,et al.  In-Class Simulation Games: Assessing Student Learning , 2003, J. Inf. Technol. Educ..

[87]  Marc Prensky,et al.  Digital game-based learning , 2000, CIE.

[88]  Steven C. Gold,et al.  Developments in Business Gaming , 2009 .

[89]  Martin Oliver,et al.  How can exploratory learning with games and simulations within the curriculum be most effectively evaluated? , 2006, Comput. Educ..

[90]  Hubert Gatignon.  Strategic studies in Markstrat , 1987 .

[91]  U. Dulleck,et al.  μ-σ Games , 2012, Games.

[92]  P. Anderson,et al.  and Future Directions Business Simulations and Cognitive Learning : Developments , Desires , 2009 .