Empirical Support for a Causal Relationship Between Gamification and Learning Outcomes

Preparing for exams is an important yet stressful time for many students. Self-testing is known to be an effective preparation strategy, yet some students lack motivation to engage or persist in self-testing activities. Adding game elements to a platform supporting self-testing may increase engagement and, by extension, exam performance. We conduct a randomized controlled experiment (n=701) comparing the effect of two game elements -- a points system and a badge system -- used individually and in combination. We find that the badge system elicits significantly higher levels of voluntary self-testing activity and this effect is particularly pronounced amongst a relatively small cohort. Importantly, this increased activity translates to a significant improvement in exam scores. Our data supports a causal relationship between gamification and learning outcomes, mediated by self-testing behavior. This provides empirical support for Landers' theory of gamified learning when the gamified activity is conducted prior to measuring learning outcomes.

[1]  John Brown,et al.  Gamification » Blog Archive » Gamification in Education: What, How, Why Bother? , 2014 .

[2]  Matthew W. Lewis,et al.  Self-Explonations: How Students Study and Use Examples in Learning to Solve Problems , 1989, Cogn. Sci..

[3]  Marc Jay,et al.  Using crowdsourcing to support pro-environmental community activism , 2013, CHI.

[4]  Joaquim A. Jorge,et al.  Engaging Engineering Students with Gamification , 2013, 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES).

[5]  Sharon Tkacz,et al.  Student study techniques and the generation effect. , 1994 .

[6]  Sebastian Deterding,et al.  Gamification: designing for motivation , 2012, INTR.

[7]  Klaus Opwis,et al.  Informational, but not Intrinsically Motivating Gamification': Preliminary Findings , 2016, CHI PLAY.

[8]  Juho Hamari,et al.  Gamification in Crowdsourcing: A Review , 2016, 2016 49th Hawaii International Conference on System Sciences (HICSS).

[9]  FoxJesse,et al.  Assessing the effects of gamification in the classroom , 2015 .

[10]  Barry J. Schwartz,et al.  Effect of question production and answering on prose recall. , 1975 .

[11]  Patrick Buckley,et al.  Gamification and student motivation , 2016, Interact. Learn. Environ..

[12]  Seiji Isotani,et al.  A systematic mapping on gamification applied to education , 2014, SAC.

[13]  Panagiotis G. Ipeirotis,et al.  Quizz: targeted crowdsourcing with a billion (potential) users , 2014, WWW.

[14]  Joseph S. Valacich,et al.  Enhancing the Motivational Affordance of Information Systems: The Effects of Real-Time Performance Feedback and Goal Setting in Group Collaboration Environments , 2010, Manag. Sci..

[15]  Ari Korhonen,et al.  The Effect of Achievement Badges on Students' Behavior: An Empirical Study in a University-Level Computer Science Course , 2015, Int. J. Emerg. Technol. Learn..

[16]  Chrysanne Di Marco,et al.  Towards Personality-driven Persuasive Health Games and Gamified Systems , 2017, CHI.

[17]  D. A. Kenny,et al.  The moderator-mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. , 1986, Journal of personality and social psychology.

[18]  Kenneth G. Brown,et al.  Using computers to deliver training: Which employees learn and why? , 2001 .

[19]  Joaquim A. Jorge,et al.  Improving participation and learning with gamification , 2013, Gamification.

[20]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[21]  A. C. Butler,et al.  The critical role of retrieval practice in long-term retention , 2011, Trends in Cognitive Sciences.

[22]  P. Shrout,et al.  Mediation in experimental and nonexperimental studies: new procedures and recommendations. , 2002, Psychological methods.

[23]  Juho Hamari,et al.  Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification , 2014, 2014 47th Hawaii International Conference on System Sciences.

[24]  Robb Lindgren,et al.  Using badges for shaping interactions in online learning environments , 2012, 2012 IEEE International Professional Communication Conference.

[25]  Richard N. Landers,et al.  Casual Social Games as Serious Games: The Psychology of Gamification in Undergraduate Education and Employee Training , 2011, Serious Games and Edutainment Applications.

[26]  Deborah I. Fels,et al.  Gamification in theory and action: A survey , 2015, Int. J. Hum. Comput. Stud..

[27]  Rosta Farzan,et al.  Results from deploying a participation incentive mechanism within the enterprise , 2008, CHI.

[28]  R. Landers.  Developing a Theory of Gamified Learning , 2014 .

[29]  Galia Angelova,et al.  Gamification in Education: A Systematic Mapping Study , 2015, J. Educ. Technol. Soc..

[30]  Amy K. Landers,et al.  An Empirical Test of the Theory of Gamified Learning , 2014 .

[31]  Rachel C. Callan,et al.  How to Avoid the Dark Side of Gamification: Ten Business Scenarios and Their Unintended Consequences , 2015 .

[32]  Edward L. Deci,et al.  Beyond the intrinsic-extrinsic dichotomy: Self-determination in motivation and learning , 1992 .

[33]  Albert T. Corbett,et al.  Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments , 2008 .

[34]  Michelene T. H. Chi,et al.  Eliciting Self-Explanations Improves Understanding , 1994, Cogn. Sci..

[35]  Christopher Cheong,et al.  Quick Quiz: A Gamified Approach for Enhancing Learning , 2013, PACIS.

[36]  N. J. Slamecka,et al.  The Generation Effect: Delineation of a Phenomenon , 1978 .

[37]  Alexandre N. Tuch,et al.  Towards understanding the effects of individual gamification elements on intrinsic motivation and performance , 2017, Comput. Hum. Behav..

[38]  J. Dunlosky,et al.  Strengthening the Student Toolbox: Study Strategies to Boost Learning. , 2013 .

[39]  Xing Yu,et al.  Designing Leaderboards for Gamification: Perceived Differences Based on User Ranking, Application Domain, and Personality Traits , 2017, CHI.

[40]  Tovi Grossman,et al.  GamiCAD: a gamified tutorial system for first time autocad users , 2012, UIST.

[41]  Alexandre N. Tuch,et al.  Do points, levels and leaderboards harm intrinsic motivation?: an empirical analysis of common gamification elements , 2013, Gamification.

[42]  Paul Denny,et al.  The effect of virtual achievements on student engagement , 2013, CHI.

[43]  Juho Hamari,et al.  Do badges increase user activity? A field experiment on the effects of gamification , 2017, Comput. Hum. Behav..

[44]  R. Landers,et al.  An Evaluation of Gamified Training: Using Narrative to Improve Reactions and Learning , 2017 .

[45]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[46]  John Hamer,et al.  The PeerWise system of student contributed assessment questions , 2008, ACE '08.

[47]  Siobhan O'Donovan,et al.  A case study in the gamification of a university-level games development course , 2013, SAICSIT '13.

[48]  Luis de Marcos,et al.  An empirical study comparing gamification and social networking on e-learning , 2014, Comput. Educ..

[49]  E. Churchill,et al.  Badges in Social Media: A Social Psychological Perspective , 2011 .

[50]  Heinz Mandl,et al.  How gamification motivates: An experimental study of the effects of specific game design elements on psychological need satisfaction , 2017, Comput. Hum. Behav..

[51]  Zachary Fitz-Walter,et al.  A gamified mobile application for engaging new students at university orientation , 2012, OZCHI.

[52]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .