Gamification in education: a mixed-methods study of gender on computer science students’ academic performance and identity development

Underrepresentation of women in computer science (CS) increasingly demands the necessity to find and enhance current learning engagement approaches to bring more women into computing fields. Some researchers have been exploring the influence of gamification on female students as one of these possible learning engagement strategies. Gamification refers to the introduction of video game elements into non-game activities to enhance engagement and motivation. Previous studies have reported mixed results of the impact of gamification on women. In this study, we introduce SEP-CyLE (Software Engineering and Programming Cyberlearning Environment), an online gamified tool that was designed to provide supplemental computing content to students. This paper presents a convergent mixed-methods study guided by social identity theory and self-efficacy to understand women's experiences with this gamified tool. More specifically, this study explores virtual points' and leaderboards' effects on CS identity development, self-efficacy, and performance. The results show that virtual points and the leaderboard contributed to improved performance for students of all genders, suggesting that gamification is a gender-neutral learning engagement strategy that improves female students' performance as much as male students. Regardless of improved performance, most women did not actively enjoy or were motivated by the virtual points or leaderboard in SEP-CyLE. Additionally, gamification had no significant impact on CS identity development or self-efficacy constructs and had little to no impact on women's interest and engagement in the field of computing.

[1]  William C. Adams,et al.  Conducting Semi‐Structured Interviews , 2015 .

[2]  Martin Hahmann,et al.  Leveraging gamification in demand dispatch systems , 2012, EDBT-ICDT '12.

[3]  Luis de Marcos,et al.  Gamifying learning experiences: Practical implications and outcomes , 2013, Comput. Educ..

[4]  Amna Khan,et al.  Use of digital game based learning and gamification in secondary school science: The effect on student engagement, learning and gender difference , 2017, Education and Information Technologies.

[5]  Matthew Montebello,et al.  Social Interactive Learning in Multiplayer Games , 2011, Serious Games and Edutainment Applications.

[6]  Peter J. Clarke,et al.  Gamification-Based Cyber-Enabled Learning Environment of Software Testing , 2016 .

[7]  J. Banfield,et al.  Increasing Student Intrinsic Motivation And Self-Efficacy Through Gamification Pedagogy , 2014 .

[8]  Henriette Cramer,et al.  Performing a check-in: emerging practices, norms and 'conflicts' in location-sharing using foursquare , 2011, Mobile HCI.

[9]  David R. Millen,et al.  Removing gamification from an enterprise SNS , 2012, CSCW.

[10]  Kenneth J. Christensen,et al.  A Structural Equation Model Analysis of Computing Identity Sub-Constructs and Student Academic Persistence , 2018, 2018 IEEE Frontiers in Education Conference (FIE).

[11]  J. Frith.  Constructing Location, One Check-in at a Time: Examining the Practices of Foursquare Users , 2012 .

[12]  Emlyn Butterfield,et al.  Gamification for Teaching and Learning Computer Security in Higher Education , 2016, ASE @ USENIX Security Symposium.

[13]  Juho Hamari,et al.  How to design gamification? A method for engineering gamified software , 2017, Inf. Softw. Technol..

[14]  Cécile Paris,et al.  Using gamification in an online community , 2012, 8th International Conference on Collaborative Computing: Networking, Applications and Worksharing (CollaborateCom).

[15]  Ian Glover,et al.  Play As You Learn: Gamification as a Technique for Motivating Learners , 2013 .

[16]  Shahab Boumi,et al.  Application of Hidden Markov Models to quantify the impact of enrollment patterns on student performance , 2019, EDM.

[17]  G. Orosz,et al.  Are Competition and Extrinsic Motivation Reliable Predictors of Academic Cheating? , 2013, Front. Psychol..

[18]  P. Burke,et al.  The Past, Present, and Future of an Identity Theory* , 2000 .

[19]  Geoff R. Goehle.  Gamification and Web-based Homework , 2013 .

[20]  N. Brickhouse,et al.  Young women's scientific identity formation in an urban context , 2001 .

[21]  Martin Valcke,et al.  Gamification in computer programming : effects on learning, engagement, self-efficacy and intrinsic motivation , 2017 .

[22]  Paul Denny,et al.  The effect of virtual achievements on student engagement , 2013, CHI.

[23]  Philip M. Sadler,et al.  Connecting High School Physics Experiences, Outcome Expectations, Physics Identity, and Physics Career Choice: A Gender Study. , 2010 .

[24]  Aditya Johri,et al.  On the development of a professional identity: engineering persisters vs engineering switchers , 2009, 2009 39th IEEE Frontiers in Education Conference.

[25]  J. McGonigal.  Reality Is Broken: Why Games Make Us Better and How They Can Change the World , 2011 .

[26]  Shaghayegh Sahebi,et al.  Detecting Trait versus Performance Student Behavioral Patterns Using Discriminative Non-Negative Matrix Factorization , 2020, FLAIRS Conference.

[27]  Jennifer G. Cromley,et al.  The role of identity development, values, and costs in college STEM retention , 2014 .

[28]  Lee A. Freeman,et al.  Are Men More Technology-Oriented Than Women? The Role of Gender on the Development of General Computer Self-Efficacy of College Students , 2010, AMCIS.

[29]  Zahra Hazari,et al.  Understanding the Experiences that Contribute to the Inclusion of Underrepresented Groups in Computing , 2020, 2020 IEEE Frontiers in Education Conference (FIE).

[30]  Samira Pouyanfar,et al.  Leveraging Machine-learning Techniques to Analyze Computing Persistence in Undergraduate Programs , 2020 .

[31]  Torsten Reiners,et al.  Gamification in Education and Business , 2015 .

[32]  John W. Creswell,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2010 .

[33]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[34]  Juho Hamari,et al.  Does Gamification Work? -- A Literature Review of Empirical Studies on Gamification , 2014, 2014 47th Hawaii International Conference on System Sciences.

[35]  Patricia Kristine Sheridan,et al.  Gamification as a Strategy for Promoting Deeper Investigation in a Reverse Engineering Activity , 2012 .

[36]  Luis de Marcos,et al.  An empirical study comparing gamification and social networking on e-learning , 2014, Comput. Educ..

[37]  Kft Hew,et al.  Do points, badges and leaderboard increase learning and activity: a quasi-experiment on the effects of gamification? , 2015 .

[38]  Juho Hamari,et al.  Demographic differences in perceived benefits from gamification , 2014, Comput. Hum. Behav..

[39]  C. Steele,et al.  Ambient belonging: how stereotypical cues impact gender participation in computer science. , 2009, Journal of personality and social psychology.

[40]  Andrew N. Meltzoff,et al.  Classrooms matter: The design of virtual classrooms influences gender disparities in computer science classes , 2011, Comput. Educ..

[41]  Jing Luo,et al.  Enhancing student learning experience with technology-mediated gamification: An empirical study , 2018, Comput. Educ..

[42]  Ian Bogost,et al.  How to Do Things with Videogames , 2011 .

[43]  Kenton O'Hara,et al.  Gamification. using game-design elements in non-gaming contexts , 2011, CHI Extended Abstracts.

[44]  Bohyun Kim.  Chapter 5. Designing Gamification in the Right Way , 2015 .

[45]  Berkan Celik,et al.  A qualitative investigation of student perceptions of game elements in a gamified course , 2018, Comput. Hum. Behav..

[46]  Deborah I. Fels,et al.  Gamification in theory and action: A survey , 2015, Int. J. Hum. Comput. Stud..

[47]  Jonathan Schaeffer,et al.  Computer-game construction: A gender-neutral attractor to Computing Science , 2010, Comput. Educ..

[48]  John W. Rice,et al.  The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education , 2012, Int. J. Gaming Comput. Mediat. Simulations.

[49]  Monique S. Ross,et al.  A unicorn's tale: Examining the experiences of Black women in engineering industry , 2016 .

[50]  Brenda Cantwell Wilson,et al.  A Study of Factors Promoting Success in Computer Science Including Gender Differences , 2002, Comput. Sci. Educ..

[51]  Rosta Farzan,et al.  Results from deploying a participation incentive mechanism within the enterprise , 2008, CHI.

[52]  Shaghayegh Sahebi,et al.  Annotated Examples and Parameterized Exercises: Analyzing Students' Behavior Patterns , 2019, AIED.

[53]  David R. Flatla,et al.  Calibration games: making calibration tasks enjoyable by adding motivating game elements , 2011, UIST.

[54]  N. Brickhouse,et al.  What Kind of a Girl Does Science? The Construction of School Science Identities , 2000 .

[55]  Richard E. Mayer,et al.  A Computer-Based Game that Promotes Mathematics Learning More than a Conventional Approach , 2017, Int. J. Game Based Learn..

[56]  J. Cafazzo,et al.  Design of an mHealth App for the Self-management of Adolescent Type 1 Diabetes: A Pilot Study , 2012, Journal of medical Internet research.

[57]  Geoff Potvin,et al.  Identity, Critical Agency, and Engineering: An Affective Model for Predicting Engineering as a Career Choice , 2016 .

[58]  Kristin L. K. Koskey,et al.  Motivation, learning, and transformative experience: A study of deep engagement in science , 2010 .

[59]  Ari Korhonen,et al.  The Effect of Achievement Badges on Students' Behavior: An Empirical Study in a University-Level Computer Science Course , 2015, Int. J. Emerg. Technol. Learn..

[60]  Katheryn R. Christy,et al.  Leaderboards in a virtual classroom: A test of stereotype threat and social comparison explanations for women's math performance , 2014, Comput. Educ..

[61]  Viswanath Venkatesh,et al.  Why Don't Men Ever Stop to Ask for Directions? Gender, Social Influence, and Their Role in Technology Acceptance and Usage Behavior , 2000, MIS Q..

[62]  Geoff Potvin,et al.  A User Interface (UI) and User eXperience (UX) evaluation framework for cyberlearning environments in computer science and software engineering education , 2020, Heliyon.

[63]  Christopher Cunningham,et al.  Gamification by Design - Implementing Game Mechanics in Web and Mobile Apps , 2011 .

[64]  A. Bandura.  Self-efficacy: toward a unifying theory of behavioral change. , 1977, Psychological review.

[65]  Rachel S. Smith.  Guidelines for Authors of Learning Objects. , 2004 .

[66]  Jacquelyn J. Chini,et al.  Quantifying the relationship between student enrollment patterns and student performance , 2020, ArXiv.

[67]  Eduardo Figueiredo,et al.  Gamification in Software Engineering Education: An Empirical Study , 2017, 2017 IEEE 30th Conference on Software Engineering Education and Training (CSEE&T).

[68]  Tatsuo Nakajima,et al.  Gamifying intelligent environments , 2011, Ubi-MUI '11.

[69]  Marko C. J. D. van Eekelen,et al.  Gamification in educational software development , 2014, CSERC.

[70]  Frank J. Lee,et al.  Development of a game-design workshop to promote young girls' interest towards computing through identity exploration , 2017, Comput. Educ..

[71]  Robb Lindgren,et al.  Using badges for shaping interactions in online learning environments , 2012, 2012 IEEE International Professional Communication Conference.

[72]  Zahra Hazari,et al.  Establishing an Explanatory Model for Mathematics Identity. , 2015, Child development.

[73]  Erin D. Snyder,et al.  Gamification of board review: a residency curricular innovation , 2013, Medical education.

[74]  M. Ishtiaq.  Book Review Creswell, J. W. (2014). Research Design: Qualitative, Quantitative and Mixed Methods Approaches (4th ed.). Thousand Oaks, CA: Sage , 2019, English Language Teaching.

[75]  Philip Sheridan Buffum.  Leveraging Narrative Interactivity to Foster Computer Science Identities , 2015, ICER.

[76]  Julita Vassileva,et al.  Does gamification work for boys and girls?: An exploratory study with a virtual learning environment , 2015, SAC.

[77]  Tovi Grossman,et al.  GamiCAD: a gamified tutorial system for first time autocad users , 2012, UIST.

[78]  Susanne Robra-Bissantz,et al.  Gamification of online idea competitions: insights from an explorative case , 2011, GI-Jahrestagung.