The Technical Debt Dataset

Technical Debt analysis is increasing in popularity as nowadays researchers and industry are adopting various tools for static code analysis to evaluate the quality of their code. Despite this, empirical studies on software projects are expensive because of the time needed to analyze the projects. In addition, the results are difficult to compare as studies commonly consider different projects. In this work, we propose the Technical Debt Dataset, a curated set of project measurement data from 33 Java projects from the Apache Software Foundation. In the Technical Debt Dataset, we analyzed all commits from separately defined time frames with SonarQube to collect Technical Debt information and with Ptidej to detect code smells. Moreover, we extracted all available commit information from the git logs, the refactoring applied with Refactoring Miner, and fault information reported in the issue trackers (Jira). Using this information, we executed the SZZ algorithm to identify the fault-inducing and -fixing commits. We analyzed 78K commits from the selected 33 projects, detecting 1.8M SonarQube issues, 62K code smells, 28K faults and 57K refactorings. The project analysis took more than 200 days. In this paper, we describe the data retrieval pipeline together with the tools used for the analysis. The dataset is made available through CSV files and an SQLite database to facilitate queries on the data. The Technical Debt Dataset aims to open up diverse opportunities for Technical Debt research, enabling researchers to compare results on common projects.

[1]  Martin Fowler,et al.  Refactoring - Improving the Design of Existing Code , 1999, Addison Wesley object technology series.

[2]  Christian Bird,et al.  Diversity in software engineering research , 2013, ESEC/FSE 2013.

[3]  Apostolos Ampatzoglou,et al.  How do developers fix issues and pay back technical debt in the Apache ecosystem? , 2018, 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[4]  Gabriele Bavota,et al.  On the diffuseness and the impact on maintainability of code smells: a large scale empirical investigation , 2017, Empirical Software Engineering.

[5]  G. Ann Campbell,et al.  Cognitive Complexity — An Overview and Evaluation , 2018, 2018 IEEE/ACM International Conference on Technical Debt (TechDebt).

[6]  Audris Mockus,et al.  Quantifying the Effect of Code Smells on Maintenance Effort , 2013, IEEE Transactions on Software Engineering.

[7]  Alberto Sillitti,et al.  A Survey on Code Analysis Tools for Software Maintenance Prediction , 2018, SEDA.

[8]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[9]  Jing Li,et al.  The Qualitas Corpus: A Curated Collection of Java Code for Empirical Studies , 2010, 2010 Asia Pacific Software Engineering Conference.

[10]  Ward Cunningham,et al.  The WyCash portfolio management system , 1992, OOPSLA '92.

[11]  Alberto Sillitti,et al.  Analyzing Forty Years of Software Maintenance Models , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C).

[12]  Heikki Huttunen,et al.  On the Fault Proneness of SonarQube Technical Debt Violations: A comparison of eight Machine Learning Techniques , 2019, ArXiv.

[13]  Daniela Cruzes,et al.  The evolution and impact of code smells: A case study of two open source systems , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[14]  Harald C. Gall,et al.  Context is king: The developer perspective on the usage of static analysis tools , 2018, 2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER).

[15]  Davide Taibi,et al.  A Dynamical Quality Model to Continuously Monitor Software Maintenance , 2017 .

[16]  Gabriele Bavota,et al.  Landfill: An Open Dataset of Code Smells with Public Evaluation , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[17]  Davide Taibi,et al.  OpenSZZ: A Free, Open-Source, Web-Accessible Implementation of the SZZ Algorithm , 2020, 2020 IEEE/ACM 28th International Conference on Program Comprehension (ICPC).

[18]  Davide Fucci,et al.  Towards a Holistic Definition of Requirements Debt , 2019, 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[19]  Yann-Gaël Guéhéneuc,et al.  Ptidej: A Flexible Reverse Engineering Tool Suite , 2007, ICSM.

[20]  Nenad Medvidovic,et al.  Identifying Architectural Bad Smells , 2009, 2009 13th European Conference on Software Maintenance and Reengineering.

[21]  Filippo Lanubile,et al.  On Developers' Personality in Large-Scale Distributed Projects: The Case of the Apache Ecosystem , 2018, 2018 IEEE/ACM 13th International Conference on Global Software Engineering (ICGSE).

[22]  Jesús M. González-Barahona,et al.  Reproducibility and credibility in empirical software engineering: A case study based on a systematic literature review of the use of the SZZ algorithm , 2018, Inf. Softw. Technol..

[23]  Nyyti Saarimäki.  Methodological Issues in Observational Studies , 2019, SOEN.

[24]  Andrea Janes,et al.  A Continuous Software Quality Monitoring Approach for Small and Medium Enterprises , 2017, ICPE Companion.

[25]  Paris Avgeriou,et al.  The Evolution of Technical Debt in the Apache Ecosystem , 2017, ECSA.

[26]  Tracy Hall,et al.  A Systematic Literature Review on Fault Prediction Performance in Software Engineering , 2012, IEEE Transactions on Software Engineering.

[27]  Thomas J. Mowbray,et al.  AntiPatterns: Refactoring Software, Architectures, and Projects in Crisis , 1998 .

[28]  Uirá Kulesza,et al.  A Framework for Evaluating the Results of the SZZ Approach for Identifying Bug-Introducing Changes , 2017, IEEE Transactions on Software Engineering.

[29]  Antonio Martini,et al.  Towards surgically-precise technical debt estimation: early results and research roadmap , 2019, MaLTeSQuE@ESEC/SIGSOFT FSE.

[30]  Maria Teresa Baldassarre,et al.  On the Accuracy of SonarQube Technical Debt Remediation Time , 2019, 2019 45th Euromicro Conference on Software Engineering and Advanced Applications (SEAA).

[31]  Raed Shatnawi,et al.  An empirical study of the bad smells and class error probability in the post-release object-oriented system evolution , 2007, J. Syst. Softw..

[32]  Davide Spadini,et al.  PyDriller: Python framework for mining software repositories , 2018, ESEC/SIGSOFT FSE.

[33]  Alberto Bacchelli,et al.  On the Impact of Design Flaws on Software Defects , 2010, 2010 10th International Conference on Quality Software.

[34]  Davide Taibi,et al.  On the Relationship Between Coupling and Refactoring: An Empirical Viewpoint , 2019, 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[35]  Danny Dig,et al.  Accurate and Efficient Refactoring Detection in Commit History , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[36]  Thomas Zimmermann,et al.  When do changes induce fixes? On Fridays , 2005 .

[37]  Davide Taibi,et al.  An Empirical Study on Technical Debt in a Finnish SME , 2019, 2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[38]  Foutse Khomh,et al.  An exploratory study of the impact of antipatterns on class change- and fault-proneness , 2011, Empirical Software Engineering.

[39]  Michel Wermelinger,et al.  Assessing the effect of clones on changeability , 2008, 2008 IEEE International Conference on Software Maintenance.

[40]  Michele Marchesi,et al.  The JIRA Repository Dataset: Understanding Social Aspects of Software Development , 2015, PROMISE.

[41]  Davide Taibi,et al.  On the diffuseness of code technical debt in Java projects of the apache ecosystem , 2019, TechDebt@ICSE.