Fostering good coding practices through individual feedback and gamification: an industrial case study

Code quality is a constant challenge faced by today’s software industry. To ensure that developers follow good coding practices, a variety of program analysis and test coverage tools are routinely deployed. However, these tools often fail to engage and change the practices of developers when applied to legacy systems as they output a huge number of warnings, quickly overwhelming the developers. In this article, we explore how individual feedback and gamification can motivate developers to pay more attention to good coding practices. To that extent, we implemented these two concepts in a tool that we deployed at two large companies where we conducted a case study. We find out that individual feedback is essential for motivating developers. We also find that gamification can be useful but has to be used with caution as it can frustrate some developers. Finally, we reflect on some lessons learned during our case studies, and conclude that the promising approach of our tool needs to be supported by longitudinal studies as well as comparative studies.

[1]  Andrea Mocci,et al.  How to gamify software engineering , 2017, 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[2]  Audris Mockus,et al.  Test coverage and post-verification defects: A multiple case study , 2009, ESEM 2009.

[3]  Christian Bird,et al.  What developers want and need from program analysis: An empirical study , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[4]  A. Rustichini,et al.  Performance in Competitive Environments: Gender Differences , 2003 .

[5]  Bill Curtis,et al.  Estimating the Principal of an Application's Technical Debt , 2012, IEEE Software.

[6]  Keith H. Bennett,et al.  Legacy Systems: Coping with Success , 1995, IEEE Softw..

[7]  Roger Azevedo,et al.  A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction , 1995 .

[8]  Mario Piattini,et al.  Gamification in software engineering - A systematic mapping , 2015, Inf. Softw. Technol..

[9]  Fergal McCaffery,et al.  The Gamification of SPICE , 2012, SPICE.

[10]  Lael J. Schooler,et al.  The Disruptive Potential of Immediate Feedback , 2008 .

[11]  M. R. Mercer,et al.  Code coverage, what does it mean in terms of quality? , 2001, Annual Reliability and Maintainability Symposium. 2001 Proceedings. International Symposium on Product Quality and Integrity (Cat. No.01CH37179).

[12]  Arie van Deursen,et al.  Why and how JavaScript developers use linters , 2017, ASE 2017.

[13]  Andy Zaidman,et al.  Analyzing the State of Static Analysis: A Large-Scale Evaluation in Open Source Software , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[14]  Lionel C. Briand,et al.  Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria , 2006, IEEE Transactions on Software Engineering.

[15]  Emerson R. Murphy-Hill,et al.  A perspective on blending programming environments and games: Beyond points, badges, and leaderboards , 2016, 2016 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[16]  Matthias Jarke,et al.  Gamification for enforcing coding conventions , 2015, ESEC/SIGSOFT FSE.

[17]  Jeffrey K. Hollingsworth,et al.  Efficient instrumentation for code coverage testing , 2002, ISSTA '02.

[18]  Helen Sharp,et al.  Motivation in Software Engineering: A systematic literature review , 2008, Inf. Softw. Technol..

[19]  Hironori Washizaki,et al.  A Gamified Tool for Motivating Developers to Remove Warnings of Bug Pattern Tools , 2014, 2014 6th International Workshop on Empirical Software Engineering in Practice.

[20]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[21]  Deborah I. Fels,et al.  Gamification in theory and action: A survey , 2015, Int. J. Hum. Comput. Stud..

[22]  Bogdan Vasilescu,et al.  Human aspects, gamification, and social media in collaborative software engineering , 2014, ICSE Companion.

[23]  Clifford J. Maloney,et al.  Systematic mistake analysis of digital computer programs , 1963, CACM.

[24]  Jean-Louis Letouzey,et al.  Managing Technical Debt with the SQALE Method , 2012, IEEE Software.

[25]  Gordon Fraser,et al.  Code Defenders: Crowdsourcing Effective Tests and Subtle Mutants with a Mutation Testing Game , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[26]  David A. Nadler,et al.  The effects of feedback on task group behavior: A review of the experimental research , 1979 .

[27]  Tamara Munzner,et al.  Design Study Methodology: Reflections from the Trenches and the Stacks , 2012, IEEE Transactions on Visualization and Computer Graphics.

[28]  Lennart E. Nacke,et al.  From game design elements to gamefulness: defining "gamification" , 2011, MindTrek.

[29]  Leif Singer,et al.  It was a bit of a race: Gamification of version control , 2012, 2012 Second International Workshop on Games and Software Engineering: Realizing User Engagement with Game Engineering Techniques (GAS).

[30]  Emerson R. Murphy-Hill,et al.  Experiences gamifying developer adoption of practices and tools , 2014, ICSE Companion.

[31]  Fabio Q. B. da Silva,et al.  Motivation in software engineering: A systematic review update , 2011, EASE.

[32]  David Hovemeyer,et al.  Using Static Analysis to Find Bugs , 2008, IEEE Software.

[33]  Robert W. Bowdidge,et al.  Why don't software developers use static analysis tools to find bugs? , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[34]  E. Deci,et al.  A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. , 1999, Psychological bulletin.

[35]  Reid Holmes,et al.  Coverage is not strongly correlated with test suite effectiveness , 2014, ICSE.

[36]  Helen Sharp,et al.  What Do We Know about Developer Motivation? , 2008, IEEE Software.