Software developer productivity loss due to technical debt - A replication and extension study examining developers' development work

Software companies need to deliver customer value continuously, both from a short- and long-term perspective. However, software development can be impeded by technical debt (TD). Although significant theoretical work has been undertaken to describe the negative effects of TD, little empirical evidence exists on how much wasted time and additional activities TD causes. The study aims to explore the consequences of TD in terms of wastage of development time. This study investigates on which activities this wasted time is spent and whether different TD types impact the wasted time differently. This study reports the results of a longitudinal study surveying 43 developers and including16 interviews followed by validation by an additional study using a different and independent dataset and focused on replicating the findings addressing the findings. The analysis of the reported wasted time revealed that developers waste, on average, 23% of their time due to TD and that developers are frequently forced to introduce new TD. The most common activity on which additional time is spent is performing additional testing. The study provides evidence that TD hinders developers by causing an excessive waste of working time, where the wasted time negatively affects productivity.

[1]  Pekka Abrahamsson,et al.  Do feelings matter? On the correlation of affects and the self‐assessed productivity in software engineering , 2014, J. Softw. Evol. Process..

[2]  Natalia Juristo Juzgado,et al.  Basics of Software Engineering Experimentation , 2010, Springer US.

[3]  Steve Counsell,et al.  The role and value of replication in empirical software engineering results , 2018, Inf. Softw. Technol..

[4]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[5]  Jan Bosch,et al.  Technical Debt Cripples Software Developer Productivity: A Longitudinal Study on Developers’ Daily Software Development Work , 2018, 2018 IEEE/ACM International Conference on Technical Debt (TechDebt).

[6]  Richard T. Vidgen,et al.  An exploration of technical debt , 2013, J. Syst. Softw..

[7]  Jan Bosch,et al.  Managing architectural technical debt: A unified model and systematic literature review , 2018, J. Syst. Softw..

[8]  Niklas Mellegård,et al.  Using weekly open defect reports as an indicator for software process efficiency: theoretical framework and a longitudinal automotive industrial case study , 2017, IWSM-Mensura.

[9]  Yuanfang Cai,et al.  A Case Study in Locating the Architectural Roots of Technical Debt , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[10]  Jeffrey C. Carver,et al.  The role of replications in Empirical Software Engineering , 2008, Empirical Software Engineering.

[11]  Carolyn B. Seaman,et al.  Qualitative Methods in Empirical Studies of Software Engineering , 1999, IEEE Trans. Software Eng..

[12]  Peng Liang,et al.  A systematic mapping study on technical debt and its management , 2015, J. Syst. Softw..

[13]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[14]  Jan Bosch,et al.  On the interest of architectural technical debt: Uncovering the contagious debt phenomenon , 2017, J. Softw. Evol. Process..

[15]  Walt Scacchi,et al.  UNDERSTANDING SOFTWARE PRODUCTIVITY , 1995 .

[16]  Jan Bosch,et al.  The Pricey Bill of Technical Debt: When and by Whom will it be Paid? , 2017, 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[17]  Rodrigo O. Spínola,et al.  Towards an Ontology of Terms on Technical Debt , 2014, 2014 Sixth International Workshop on Managing Technical Debt.

[18]  Monalessa Perini Barcellos,et al.  An ontology-based approach for integrating tools supporting the software measurement process , 2017, Sci. Comput. Program..

[19]  Katrina Maxwell Collecting Data for Comparability: Benchmarking Software Development Productivity , 2001, IEEE Softw..

[20]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[21]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[22]  Randall W. Jensen Improving Software Development Productivity: Effective Leadership and Quantitative Methods in Software Management , 2014 .

[23]  John L. Campbell,et al.  Coding In-depth Semistructured Interviews , 2013 .

[24]  Robert E. Ployhart,et al.  Longitudinal Research: The Theory, Design, and Analysis of Change , 2010 .

[25]  Bill Curtis,et al.  Estimating the size, cost, and types of Technical Debt , 2012, 2012 Third International Workshop on Managing Technical Debt (MTD).

[26]  Barbara A. Kitchenham,et al.  Combining empirical results in software engineering , 1998, Inf. Softw. Technol..

[27]  Robert L. Nord,et al.  Technical Debt: From Metaphor to Theory and Practice , 2012, IEEE Software.

[28]  Peng Liang,et al.  Architectural Technical Debt Identification Based on Architecture Decisions and Change Scenarios , 2015, 2015 12th Working IEEE/IFIP Conference on Software Architecture.

[29]  M. Murray,et al.  Critical Reflection: A Textbook for Critical Thinking , 2005 .

[30]  R. Yin Case Study Research: Design and Methods , 1984 .

[31]  Jan Bosch,et al.  Technical Debt tracking: Current state of practice: A survey and multiple case study in 15 large organizations , 2018, Sci. Comput. Program..

[32]  Neil A. Ernst,et al.  Measure it? Manage it? Ignore it? software practitioners and technical debt , 2015, ESEC/SIGSOFT FSE.

[33]  Katrina Maxwell Software development productivity , 2003, Adv. Comput..

[34]  Robert J. Eisenberg A threshold based approach to technical debt , 2012, SOEN.

[35]  Peng Liang,et al.  Architectural Debt Management in Value-Oriented Architecting , 2013, Economics-Driven Software Architecture.

[36]  Jennifer Pérez,et al.  Guiding Flexibility Investment in Agile Architecting , 2014, 2014 47th Hawaii International Conference on System Sciences.

[37]  Ipek Ozkaya,et al.  Managing Technical Debt in Software Engineering (Dagstuhl Seminar 16162) , 2016, Dagstuhl Reports.

[38]  Forrest Shull,et al.  Identification and management of technical debt: A systematic mapping study , 2016, Inf. Softw. Technol..

[39]  Ward Cunningham,et al.  The WyCash portfolio management system , 1992, OOPSLA '92.

[40]  Jan Bosch,et al.  Time to Pay Up: Technical Debt from a Software Quality Perspective , 2017, CIbSE.

[41]  Natalia Juristo Juzgado,et al.  Analysis of the influence of communication between researchers on experiment replication , 2006, ISESE '06.

[42]  Davi Viana,et al.  How have Software Engineering Researchers been Measuring Software Productivity? - A Systematic Mapping Study , 2017, ICEIS.

[43]  Donald F. Morrison,et al.  The Optimal Spacing of Repeated Measurements , 1970 .

[44]  Yuanfang Cai,et al.  Using technical debt data in decision making: Potential decision approaches , 2012, 2012 Third International Workshop on Managing Technical Debt (MTD).

[45]  Per Runeson,et al.  Guidelines for conducting and reporting case study research in software engineering , 2009, Empirical Software Engineering.

[46]  Robert L. Nord,et al.  Reducing Friction in Software Development , 2016, IEEE Software.

[47]  Shari Lawrence Pfleeger,et al.  Preliminary Guidelines for Empirical Research in Software Engineering , 2002, IEEE Trans. Software Eng..

[48]  Paul Ralph,et al.  Software Development Waste , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[49]  Jan Bosch,et al.  Architecture Technical Debt: Understanding Causes and a Qualitative Model , 2014, 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications.

[50]  James Miller,et al.  Triangulation as a basis for knowledge discovery in software engineering , 2008, Empirical Software Engineering.

[51]  H. D. Rombach,et al.  The Goal Question Metric Approach , 1994 .