Is It Worth Using Gamification on Software Testing Education? An Extended Experience Report in the Context of Undergraduate Students

Context: Testing is fundamental in the software development process. Nevertheless, testing education faces the key challenge of ensuring that undergraduate students acquire knowledge and skills they need for their future careers by matching what is taught in the classroom to industry standards. In this context, gamification can be used as an alternative educational approach. It uses game elements in real-world contexts in order to increase people's motivation and engagement in tasks that require external stimuli, especially in educational contexts.Objective: Reporting on results of an experimental study designed to assess the impact of gamification on software testing education, as well as reporting on the experience of building a supporting gamified platform. Method: We performed a systematic literature mapping aiming at characterizing how gamification has been explored in the software testing context. In addition, some of the problems faced by testing education were identified through an ad-hoc search. Then, we developed a gamified approach and a platform that have been used to run five 4-hour functional testing classes with undergraduate students from four Brazilian institutions of higher education.Results: Overall, students who learned with the traditional approach felt more motivated than those who learned with the gamified approach, whereas the performance of both groups were similar. On the other hand, feedback questionnaires indicated that the gamified class was more attractive (in terms of attention) and funny. Moreover, we observed that building a gamified platform is complex and challenging, particularly for the definition of the game mechanics and dynamics.Conclusion: Even though the results in terms of students' performance was neutral, and the motivation of students did not increase due to the introduction of gamification, the experience of having used this alternative approach is considered positive, as it provided a more enjoyable and funny environment, both from the researcher's and students' point of view. Furthermore,  with this experience we foresee we can do better in terms of gamified teaching in future work.

[1]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[2]  Richard J. Lipton,et al.  Hints on Test Data Selection: Help for the Practicing Programmer , 1978, Computer.

[3]  K. Werbach,et al.  For the Win: How Game Thinking Can Revolutionize Your Business , 2012 .

[4]  Gail E. Kaiser,et al.  Increasing Student Engagement in Software Engineering with Gamification , 2012 .

[5]  Gordon Fraser,et al.  Teaching Software Testing Concepts Using a Mutation Testing Game , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET).

[6]  Giordano Tamburrelli,et al.  Understanding gamification mechanisms for software development , 2013, ESEC/FSE 2013.

[7]  Mahdi Houshmand,et al.  EMVille: A gamification-based approach to address the equivalent mutant problem , 2017, 2017 7th International Conference on Computer and Knowledge Engineering (ICCKE).

[8]  Claes Wohlin,et al.  Experimentation in Software Engineering , 2000, The Kluwer International Series in Software Engineering.

[9]  Gordon Fraser,et al.  Code Defenders: A Mutation Testing Game , 2016, 2016 IEEE Ninth International Conference on Software Testing, Verification and Validation Workshops (ICSTW).

[10]  Brauner R. N. Oliveira,et al.  Can we use the Flipped Classroom Model to teach Black-box Testing to Computer Students? , 2019, SBQS.

[11]  Félix García,et al.  A framework for gamification in software engineering , 2017, J. Syst. Softw..

[12]  Tor Stålhane,et al.  Challenges and strategies for motivating software testing personnel , 2016, Inf. Softw. Technol..

[13]  Julita Vassileva,et al.  Does gamification work for boys and girls?: An exploratory study with a virtual learning environment , 2015, SAC.

[14]  Edward L. Deci,et al.  Levels of Analysis, Regnant Causes of Behavior and Well-Being: The Role of Psychological Needs , 2011 .

[15]  Matheus Marabesi,et al.  Towards a Gamified Tool to Improve Unit Test Teaching , 2019, 2019 XIV Latin American Conference on Learning Technologies (LACLO).

[16]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[17]  Ellen Francine Barbosa,et al.  Educational games: A contribution to software testing education , 2017, 2017 IEEE Frontiers in Education Conference (FIE).

[18]  Jana Schumann,et al.  Confounding parameters on program comprehension: a literature survey , 2015, Empirical Software Engineering.

[19]  Jonathan Bell,et al.  Secret ninja testing with HALO software engineering , 2011, SSE '11.

[20]  Gordon Fraser,et al.  Gamifying a Software Testing Course with Code Defenders , 2019, SIGCSE.

[21]  Fabiano Cutigi Ferrari,et al.  Gamification in Software Testing: A Characterization Study , 2018, SAST '18.

[22]  Luciano Baresi,et al.  An Introduction to Software Testing , 2006, FoVMT.

[23]  Sandro Ronaldo Bezerra Oliveira,et al.  A Systematic Strategy to Teaching of Exploratory Testing using Gamification , 2019, ENASE.

[24]  Peter J. Clarke,et al.  Gamification-Based Cyber-Enabled Learning Environment of Software Testing , 2016 .

[25]  Renée A. McCauley,et al.  Facilitating Programming Success in Data Science Courses through Gamified Scaffolding and Learn2Mine , 2015, ITiCSE.

[26]  Fabrício de Sousa Pinto,et al.  Gamification applied for Software Engineering teaching-learning process , 2017, SBES'17.

[27]  Gordon Fraser,et al.  Code Defenders: Crowdsourcing Effective Tests and Subtle Mutants with a Mutation Testing Game , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[28]  Patrícia Augustin Jaques,et al.  Infographics or Graphics+Text: Which Material is Best for Robust Learning? , 2016, 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT).

[29]  Mary Jean Harrold,et al.  Testing: a roadmap , 2000, ICSE '00.

[30]  Fabiano Cutigi Ferrari,et al.  Is It Worth Using Gamification on Software Testing Education?: An Experience Report , 2019, SBQS.

[31]  Jean Felipe P. Cheiran,et al.  Problem-Based Learning to Align Theory and Practice in Software Testing Teaching , 2017, SBES'17.

[32]  Fabiane Barreto Vavassori Benitti As a teacher, I want to know what to teach in requirements engineering so that professionals can be better prepared , 2017, SBES'17.

[33]  Gordon Fraser,et al.  Teaching Mutation Testing using Gamification , 2016 .

[34]  Joanna Smith,et al.  Using peer review to teach software testing , 2012, ICER '12.

[35]  Sandro Ronaldo Bezerra Oliveira,et al.  Gamification and Evaluation of the Use the Agile Tests in Software Quality Subjects: The Application of Case Studies , 2018, ENASE.