A meta-analytic review of the role of instructional support in game-based learning

Computer games can be considered complex learning environments in which players require instructional support to engage in cognitive processes such as selecting and actively organizing/integrating new information. We used meta-analytical techniques to test if instructional support enhances learning in game-based learning (k = 107, N"a"d"j = 3675). We found that instructional support in game-based learning environments improved learning (d = .34, p < .001). Additional moderator analyses revealed that the learning effect was largest when learning of skills was involved (d = .62, p < .001) and when the instructional support aimed at the selection of relevant new information (d = .46, p < .001). Furthermore, we found some evidence for a publication bias since the effect sizes for studies in peer-reviewed journals was significantly higher than for studies in proceedings and unpublished studies (journals: d = .44; proceedings: d = .08; unpublished: d = .14).

[1]  John V. Dempsey,et al.  Instructional Gaming: Implications for Instructional Technology. , 1994 .

[2]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[3]  Jeffrey A. Jordan,et al.  Competition and Performance on a Computer-Based Complex Perceptual-Motor Task , 1997, Hum. Factors.

[4]  R. Mayer,et al.  Learning Science in Virtual Reality Multimedia Environments: Role of Methods and Media , 2002 .

[5]  James C. Lester,et al.  The Case for Social Agency in Computer-Based Teaching: Do Students Learn More Deeply When They Interact With Animated Pedagogical Agents? , 2001 .

[6]  Michelle F. Wright,et al.  Computer Gaming and Interactive Simulations for Learning: A Meta-Analysis , 2006 .

[7]  Wayne L. Shebilske,et al.  Effects of Group Discussion on Acquisition of Complex Skills , 1996, Hum. Factors.

[8]  Rob Nadolski,et al.  Scripted collaboration in serious gaming for complex learning: Effects of multiple perspectives when acquiring water management skills , 2011, Br. J. Educ. Technol..

[9]  H. H. Leemkuil,et al.  Is it all in the game? Learner support in an educational knowledge management simulation game , 2006 .

[10]  Traci Sitzmann.  A META-ANALYTIC EXAMINATION OF THE INSTRUCTIONAL EFFECTIVENESS OF COMPUTER-BASED SIMULATION GAMES , 2011 .

[11]  Hans Hoeken,et al.  Suspense, curiosity, and surprise: How discourse structure influences the affective and cognitive processing of a story , 2000 .

[12]  F. Paas,et al.  How to Optimize Learning From Animated Models: A Review of Guidelines Based on Cognitive Load , 2008 .

[13]  Patricia D. Mautone,et al.  Pictorial aids for learning by doing in a multimedia geology simulation game. , 2002 .

[14]  Thomas W. Malone,et al.  Toward a Theory of Intrinsically Motivating Instruction , 1981, Cogn. Sci..

[15]  Robert L. Goldstone,et al.  Transformational Play as a Curricular Scaffold: Using Videogames to Support Science Education , 2009 .

[16]  Francis M. Dwyer,et al.  The Effect of Online Gaming, Cognition and Feedback Type in Facilitating Delayed Achievement of Different Learning Objectives , 2005 .

[17]  Herre van Oostendorp,et al.  Current Practices in Serious Game Research: A Review from a Learning Outcomes Perspective , 2009 .

[18]  E. Salas,et al.  Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. , 1993 .

[19]  W. Kintsch.  Learning from text, levels of comprehension, or: Why anyone would read a story anyway , 1980 .

[20]  Naomi J. Aldrich,et al.  Does Discovery-Based Instruction Enhance Learning?. , 2011 .

[21]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .

[22]  Ton de Jong,et al.  Scientific Discovery Learning with Computer Simulations of Conceptual Domains , 1998 .

[23]  Cheryl I. Johnson,et al.  Applying the self-explanation principle to multimedia learning in a computer-based game-like environment , 2010, Comput. Hum. Behav..

[24]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[25]  R. Mayer,et al.  Narrative games for learning: Testing the discovery and narrative hypotheses. , 2012 .

[26]  Richard N. Van Eck,et al.  The effect of competition and contextualized advisement on the transfer of mathematics skills a computer-based instructional simulation game , 2002 .

[27]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[28]  R. Mayer,et al.  Personalized messages that promote science learning in virtual environments , 2004 .

[29]  D. Leutner.  Guided discovery learning with computer-based simulation games: Effects of adaptive and non-adaptive instructional support , 1993 .

[30]  Richard E. Mayer,et al.  Multimedia learning and games. , 2011 .

[31]  Sam Solaimani,et al.  Conditions for learning from animations , 2008, ICLS.

[32]  Cristina Conati,et al.  Building and evaluating an intelligent pedagogical agent to improve the effectiveness of an educational game , 2004, IUI '04.

[33]  Eric Anthony Day,et al.  Collaborative Training With a More Experienced Partner: Remediating Low Pretraining Self-Efficacy in Complex Skill Acquisition , 2007, Hum. Factors.

[34]  F. Fischer,et al.  Epistemic and social scripts in computer–supported collaborative learning , 2005 .

[35]  James E. Driskell,et al.  Games, Motivation, and Learning: A Research and Practice Model , 2002 .

[36]  Neil Salkind,et al.  Encyclopedia of research design , 2010 .

[37]  R. Mayer.  Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. , 2004, The American psychologist.

[38]  Jeffrey A. Jordan,et al.  A Dyadic Protocol for Training Complex Skills , 1992 .

[39]  H. O'Neil,et al.  Classification of learning outcomes: evidence from the computer games literature , 2005 .

[40]  T. Trabasso,et al.  Constructing inferences during narrative text comprehension. , 1994 .

[41]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[42]  R. Mayer.  Applying the science of learning: evidence-based principles for the design of multimedia instruction. , 2008, The American psychologist.

[43]  Marc Prensky,et al.  Digital game-based learning , 2000, CIE.

[44]  R. Mayer,et al.  Role of guidance, reflection, and interactivity in an agent-based multimedia game , 2005 .

[45]  M. Lepper,et al.  Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. , 1996 .

[46]  W. Shebilske,et al.  Interlesson spacing and task-related processing during complex skill acquisition. , 1999 .

[47]  Jeffrey A. Jordan,et al.  Dyadic Versus Individual Training Protocols: Loss and Reacquisition of a Complex Skill , 1997 .

[48]  M. P. Jacob Habgood,et al.  Motivating Children to Learn Effectively: Exploring the Value of Intrinsic Integration in Educational Games , 2011 .

[49]  Fengfeng Ke,et al.  A Qualitative Meta-Analysis of Computer Games as Learning Tools , 2009 .

[50]  Robert Rosenthal.  WRITING META-ANALYTIC REVIEWS , 1995 .

[51]  Herre van Oostendorp,et al.  The role of Game Discourse Analysis and curiosity in creating engaging and effective serious games by implementing a back story and foreshadowing , 2011, Interact. Comput..

[52]  Paul Ginns.  Meta-Analysis of the Modality Effect. , 2005 .

[53]  James C. Lester,et al.  Enhancing 5th graders' science content knowledge and self-efficacy through game-based learning , 2011, Comput. Educ..

[54]  Bob J. Wielinga,et al.  The role of prescriptive models in learning , 2012, Comput. Educ..

[55]  Hans van der Meij,et al.  Learning from games: Does collaboration help? , 2011, Br. J. Educ. Technol..

[56]  Harold F. O'Neil,et al.  The effect of presenting worked examples for problem solving in a computer game , 2007 .

[57]  Fran C. Blumberg.  The effects of children's goals for learning on video game performance , 2000 .

[58]  Kori Inkpen Quinn,et al.  Playing together beats playing apart, especially for girls , 1995, CSCL.

[59]  Jonathan P. Rowe,et al.  Story-Based Learning: The Impact of Narrative on Learning Experiences and Outcomes , 2008, Intelligent Tutoring Systems.

[60]  Michael Zyda,et al.  From visual simulation to virtual reality to games , 2005, Computer.

[61]  F. Paas,et al.  Cognitive Load Theory and Instructional Design: Recent Developments , 2003 .

[62]  Paul D. Ellis,et al.  The essential guide to effect sizes : statistical power, meta-analysis, and the interpretation of research results , 2010 .

[63]  Walter Kintsch,et al.  Comprehension: A Paradigm for Cognition , 1998 .

[64]  P. Wouters,et al.  A meta-analysis of the cognitive and motivational effects of serious games , 2013 .

[65]  Hua Wang,et al.  Multimodality and Interactivity: Connecting Properties of Serious Games with Educational Outcomes , 2009, Cyberpsychology Behav. Soc. Netw..

[66]  Ton de Jong,et al.  Instructional support in games , 2011, CGAMES 2011.

[67]  R. Mayer,et al.  Nine Ways to Reduce Cognitive Load in Multimedia Learning , 2003 .