Onboarding inexperienced developers: struggles and perceptions regarding automated testing

Previous research found that inexperienced software engineers may tend to view automatic testing as a waste of time and as an activity completely separate from programming. This could have a negative impact on their later careers and could be a sign that improvements in software engineering education are needed when it comes to testing. At the same time, this stance could negatively influence the perception that practitioners have of recent university graduates. To explore this issue, we conducted a qualitative study and surveyed 170 and interviewed 22 practitioners about their experiences with recent graduates, focusing on software testing skills. We find that practitioners do recognize a skill gap between university graduates and industry expectations and that this perception could be engrained deeply enough already to influence hiring practices. Practitioners use different and at times costly strategies to alleviate this skill gap, such as training and mentoring efforts. We validated core findings in a survey with 698 professional software developers. Our qualitative insights can help industry, research, and educational institutions guide in-depth studies that explore the severity of the effects we have found. The coping strategies we have found can provide valuable starting points that can inform changes in how we educate the software engineers of the future.

[1]  E. L. Jones An experiential approach to incorporating software testing into the computer science curriculum , 2001, 31st Annual Frontiers in Education Conference. Impact on Engineering and Science Education. Conference Proceedings (Cat. No.01CH37193).

[2]  Dean Knudson,et al.  Investigating the skill gap between graduating students and industry expectations , 2014, ICSE Companion.

[3]  Amber Settle,et al.  Testing first: emphasizing testing in early programming courses , 2005, ITiCSE '05.

[4]  C. Brodsky The Discovery of Grounded Theory: Strategies for Qualitative Research , 1968 .

[6]  Maggie Johnson,et al.  Learning to Be a Programmer in a Complex Organization: A Case Study on Practice-Based Learning during the Onboarding Process at Google , 2010 .

[7]  Stephen H. Edwards Rethinking computer science education from a test-first perspective , 2003, OOPSLA '03.

[8]  Cem Kaner,et al.  Practice and Transfer of Learning in the Teaching of Software Testing , 2007, 20th Conference on Software Engineering Education & Training (CSEET'07).

[9]  Stephen H. EDWARDS Using Test-Driven Development in the Classroom : Providing Students with Automatic , Concrete Feedback on Performance , 2003 .

[10]  Andrew Begel,et al.  Novice software developers, all over again , 2008, ICER '08.

[11]  Kurt Schneider,et al.  Communicating software testing culture through visualizing testing activity , 2015, SSE@SIGSOFT FSE.

[12]  David S. Janzen,et al.  Test-driven learning: intrinsic integration of testing into the CS/SE curriculum , 2006, SIGCSE '06.

[13]  Margaret-Anne D. Storey,et al.  The Emergence of GitHub as a Collaborative Platform for Education , 2015, CSCW.

[14]  Roberta Coelho,et al.  POPT: A Problem-Oriented Programming and Testing approach for novice students , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[15]  Kurt Schneider,et al.  Automatically recommending test code examples to inexperienced developers , 2015, ESEC/SIGSOFT FSE.

[16]  Edward L. Jones,et al.  A perspective on teaching software testing , 2001 .

[17]  Leif Singer,et al.  Enablers, inhibitors, and perceptions of testing in novice software teams , 2014, SIGSOFT FSE.

[18]  Christoph Treude,et al.  Mutual assessment in the social programmer ecosystem: an empirical investigation of developer profile aggregators , 2013, CSCW.

[19]  Tom DeMarco,et al.  Peopleware: Productive Projects and Teams , 1987 .

[20]  Kurt Schneider,et al.  Serious Insights Through Fun Software-Projects , 2004, EuroSPI.

[21]  Rachel K. E. Bellamy,et al.  Moving into a new software project landscape , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[22]  James A. Whittaker,et al.  How Google tests software , 2012, CSIIRW '10.

[23]  Sebastian G. Elbaum,et al.  Bug Hunt: Making Early Software Testing Lessons Engaging and Affordable , 2007, 29th International Conference on Software Engineering (ICSE'07).

[24]  Eric Brechner Things they would not teach me of in college: what Microsoft developers learn later , 2003, OOPSLA '03.

[25]  Kwee Heong Tan How Google tests software by James A. Whittaker, Jason Arbon and Jeff Carollo , 2012, SOEN.

[26]  Andrew Begel,et al.  Struggles of new college graduates in their first software development job , 2008, SIGCSE '08.

[27]  Fabian Fagerholm,et al.  Onboarding in Open Source Software Projects: A Preliminary Analysis , 2013, 2013 IEEE 8th International Conference on Global Software Engineering Workshops.

[28]  Susan Elliott Sim,et al.  The ramp-up problem in software projects: a case study of how software immigrants naturalize , 1998, Proceedings of the 20th International Conference on Software Engineering.

[29]  Leif Singer,et al.  Creating a shared understanding of testing culture on a social coding site , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[30]  Diane Kelly,et al.  More testing should be taught , 2001, CACM.

[31]  Laura A. Dabbish,et al.  Social transparency in networked information exchange: a theoretical framework , 2012, CSCW.

[32]  Robert L. Glass,et al.  Software Testing and Industry Needs , 2006, IEEE Softw..

[33]  Vahid Garousi,et al.  Developing, Verifying, and Maintaining High-Quality Automated Test Scripts , 2016, IEEE Software.

[34]  Soon-Kyeong Kim,et al.  Teaching software design with open source software , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[35]  Tsong Yueh Chen,et al.  A preliminary survey on software testing practices in Australia , 2004, 2004 Australian Software Engineering Conference. Proceedings..

[36]  Ioannis Stamelos,et al.  A Framework for Teaching Software Testing using F/OSS Methodology , 2006, OSS.

[37]  William Pugh,et al.  Helping students appreciate test-driven development (TDD) , 2006, OOPSLA '06.

[38]  Bernd Brügge,et al.  Teaching on Industry-Oriented Software Engineering Course , 1992, CSEE.

[39]  E. Rogers,et al.  Diffusion of Innovations , 1964 .

[40]  David Boud,et al.  The Challenge of Problem Based Learning , 1991 .

[41]  Fabian Fagerholm,et al.  The role of mentoring and project characteristics for onboarding in open source software projects , 2014, ESEM '14.

[42]  Václav Rajlich,et al.  Teaching developer skills in the first software engineering course , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[43]  Marco Aurélio Gerosa,et al.  The hard life of open source software project newcomers , 2014, CHASE.