To gamify or not to gamify? An experimental field study of the influence of badges on motivation, activity, and performance in an online learning course

Over the last few years, the implementation of game elements like badges in non-game environments has become increasingly popular (Butler, 2014). In this study, we tested whether badges, which could be received for successful task performance and specific activities within an e-learning course in a higher education setting, had an impact on students' motivation and performance. In a between-subjects experimental field study, students were randomly assigned to three different conditions (no badges, badges visible to peers, badges only visible to students themselves). The results show that badges have less impact on motivation and performance than is commonly assumed. Independent of condition, students intrinsic motivation decreased over time. Contrary to expectation, the badges that could only be viewed by the students themselves were evaluated more positively than those that could also be viewed by others. Field experiment shows potential influences of badges in online course over time.Badges have less impact on motivation and performance than is commonly assumed.Badges do not hinder a decrease of students' intrinsic motivation over time.Badges did not seem to enable social comparison processes.

[1]  L. Festinger.  A Theory of Social Comparison Processes , 1954 .

[2]  Alexandre N. Tuch,et al.  Do points, levels and leaderboards harm intrinsic motivation?: an empirical analysis of common gamification elements , 2013, Gamification.

[3]  Edward L. Deci,et al.  The undermining effect is a reality after all--Extrinsic rewards , 1999 .

[4]  Ari Korhonen,et al.  How (not) to introduce badges to online exercises , 2014, SIGCSE.

[5]  Daniel T. Hickey,et al.  A multilevel analysis of the effects of external rewards on elementary students' motivation, engagement and learning in an educational game , 2014, Comput. Educ..

[6]  Luis de Marcos,et al.  Gamifying learning experiences: Practical implications and outcomes , 2013, Comput. Educ..

[7]  Duane T. Wegener,et al.  Evaluating the use of exploratory factor analysis in psychological research. , 1999 .

[8]  Ari Korhonen,et al.  Empirical Study on the Effect of Achievement Badges in TRAKLA2 Online Learning Environment , 2013, 2013 Learning and Teaching in Computing and Engineering.

[9]  Joaquim A. Jorge,et al.  Engaging Engineering Students with Gamification , 2013, 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES).

[10]  Abraham P. Buunk,et al.  Social Comparison in the Classroom: A Review , 2008 .

[11]  Oren Zuckerman,et al.  Deconstructing gamification: evaluating the effectiveness of continuous measurement, virtual rewards, and social comparison for promoting physical activity , 2014, Personal and Ubiquitous Computing.

[12]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[13]  Students Engagement through Gamification in Education Gamifying Formative Assessment , 2017 .

[14]  Charles Butler,et al.  A Framework for Evaluating the Effectiveness of Gamification Techniques by Personality Type , 2014, HCI.

[15]  Siobhan O'Donovan,et al.  A case study in the gamification of a university-level games development course , 2013, SAICSIT '13.

[16]  Tobias Ley,et al.  Scaling up Learning for Sustained Impact , 2013, Lecture Notes in Computer Science.

[17]  Monica Aresta,et al.  Students' Perspectives on Badges in Educational Social Media Platforms: The Case of SAPO Campus Tutorial Badges , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[18]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[19]  E. Deci,et al.  Extrinsic Rewards and Intrinsic Motivation in Education: Reconsidered Once Again , 2001 .

[20]  Juho Hamari,et al.  Do badges increase user activity? A field experiment on the effects of gamification , 2017, Comput. Hum. Behav..

[21]  Ian Glover,et al.  Play As You Learn: Gamification as a Technique for Motivating Learners , 2013 .

[22]  E. Deci,et al.  A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. , 1999, Psychological bulletin.

[23]  Dan Pitica,et al.  The gamification of the study of electronics in dedicated e-learning platforms , 2013, Proceedings of the 36th International Spring Seminar on Electronics Technology.

[24]  J. Cameron,et al.  Reinforcement, Reward, and Intrinsic Motivation: A Meta-Analysis , 1994 .

[25]  Deborah I. Fels,et al.  Gamification in theory and action: A survey , 2015, Int. J. Hum. Comput. Stud..

[26]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[27]  Sergi Villagrasa,et al.  Gamification for learning 3D computer graphics arts , 2013 .

[28]  Jason W. Osborne,et al.  Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. , 2005 .

[29]  Jesse Fox,et al.  Assessing the effects of gamification in the classroom: A longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance , 2015, Comput. Educ..

[30]  Scott Nicholson.  A User-Centered Theoretical Framework for Meaningful Gamification , 2012 .

[31]  M. Deutsch,et al.  A study of normative and informational social influences upon individual judgement. , 1955, Journal of abnormal psychology.

[32]  Pedro J. Muñoz-Merino,et al.  An analysis of the use of badges in an educational experiment , 2016, 2016 IEEE Frontiers in Education Conference (FIE).

[33]  Erik Duval,et al.  Evaluating the Use of Open Badges in an Open Learning Environment , 2013, EC-TEL.

[34]  Pascal Huguet,et al.  Social comparison choices in the classroom: further evidence for students' upward comparison tendency and its beneficial impact on performance , 2001 .

[35]  Brian P. Coppola,et al.  Skill and will: The role of motivation and cognition in the learning of college chemistry , 2003 .

[36]  Bieke Zaman,et al.  Why Gamification Fails in Education and How to Make It Successful: Introducing Nine Gamification Heuristics Based on Self-Determination Theory , 2017, Serious Games and Edutainment Applications.

[37]  Martin Ebner,et al.  MOOCs Completion Rates and Possible Methods to Improve Retention - A Literature Review , 2014 .

[38]  Jure Leskovec,et al.  Engaging with massive online courses , 2014, WWW.

[39]  Ivica Boticki,et al.  Usage of a mobile social learning platform with virtual badges in a primary school , 2015, Comput. Educ..

[40]  Kim Flintoff,et al.  Digital badges in education , 2013, Education and Information Technologies.

[41]  Johanna Pirker,et al.  Motivational active learning: engaging university students in computer science education , 2014, ITiCSE '14.

[42]  Gilad Ravid,et al.  Personality Based Gamification: How Different Personalities perceive Gamification , 2014, ECIS.

[43]  Heinz Ulrich Hoppe,et al.  Motivating Students or Teachers? - Challenges for a Successful Implementation of Online-Learning in Industry-Related Vocational Training , 2013, EC-TEL.

[44]  Carlos Delgado Kloos,et al.  Analyzing students' intentionality towards badges within a case study using Khan academy , 2016, LAK.

[45]  Liam J. Bannon,et al.  The motivation is the message: comparing CSCL in different settings , 2002, CSCL.

[46]  Ji-Hye Park,et al.  Factors Influencing Adult Learners' Decision to Drop Out or Persist in Online Learning , 2009, J. Educ. Technol. Soc..

[47]  Zeynep Turan,et al.  Gamification and Education: Achievements, Cognitive Loads, and Views of Students , 2016, Int. J. Emerg. Technol. Learn..