How do Students Test Software Units?

We gained insight into ideas and beliefs on testing of students who finished an introductory course on programming without any formal education on testing. We asked students to fill in a small survey, to do four exercises and to fill in a second survey. We interviewed eleven of these students in semi-structured interviews, to obtain more in-depth insight. The main outcome is that students do not test systematically, while most of them think they do test systematically. One of the misconceptions we found is that most students can only think of test cases based on programming code. Even if no code was provided (black-box testing), students try to come up with code to base their test cases on.

[1]  Simone do Rocio Senger de Souza,et al.  An Experience on Applying Software Testing for Teaching Introductory Programming Courses , 2012, CLEI Electron. J..

[2]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[3]  Gary Lewandowski,et al.  Debugging: the good, the bad, and the quirky -- a qualitative analysis of novices' strategies , 2008, SIGCSE '08.

[4]  Yifat Ben-David Kolikant,et al.  Students' alternative standards for correctness , 2005, ICER '05.

[5]  Stephen H. Edwards.  Using software testing to move students from trial-and-error to reflection-in-action , 2004 .

[6]  David Ginat.  On Novice Loop Boundaries and Range Conceptions , 2004, Comput. Sci. Educ..

[7]  A. Bijlsma,et al.  Integrated Test Development: An integrated and incremental approach to write software of high quality , 2018, CSERC '18.

[8]  Fabiano Cutigi Ferrari,et al.  The impact of Software Testing education on code reliability: An empirical assessment , 2017, J. Syst. Softw..

[9]  Laura M. Leventhal,et al.  Analyses of factors related to positive test bias in software testing , 1994, Int. J. Hum. Comput. Stud..

[10]  Stephen H. Edwards,et al.  Do student programmers all tend to write the same software tests? , 2014, ITiCSE '14.

[11]  Rogério Eduardo Garcia,et al.  Challenges to integrate software testing into introductory programming courses , 2017, 2017 IEEE Frontiers in Education Conference (FIE).

[12]  Natalia Juristo Juzgado,et al.  Impact of CS Programs on the Quality of Test Cases Generation: An Empirical Study , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[13]  Ralf Romeike,et al.  Addressing Teaching Practices Regarding Software Quality: Testing and Debugging in the Classroom , 2017, WiPSCE.

[14]  Leif Singer,et al.  Onboarding inexperienced developers: struggles and perceptions regarding automated testing , 2016, Software Quality Journal.

[15]  Ibrahim Cetin,et al.  Students’ Understanding of Loops and Nested Loops in Computer Programming: An APOS Theory Perspective , 2015 .