Towards Better Understanding of Software Quality Evolution through Commit-Impact Analysis

Developers intend to improve the quality of the software as it evolves. However, as software becomes larger and more complex, those intended actions may lead to unintended consequences. Analyzing change in software quality among different releases overlooks fine-grained changes that each commit introduces. We believe that studying software quality before and after each commit (commit-impact analysis) can reveal a wealth of information about how the software evolves and how each change impacts its quality. In this paper, we explore whether each commit has an impact on the source code, investigate the compilability of each impactful commit, examine how source code changes affect software quality metrics, and study the effectiveness of using a certain metric as software quality indicator. We analyze a total of 19,580 commits from 38 Apache Java software systems to better understand how change occurs, why, and by who.

[1]  James A. Kupsch,et al.  Manual vs. Automated Vulnerability Assessment: A Case Study , 2009 .

[2]  Hridesh Rajan,et al.  Boa: A language and infrastructure for analyzing ultra-large-scale software repositories , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[3]  Themistoklis G. Diamantopoulos,et al.  QualBoa: Reusability-aware Recommendations of Source Code Components , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[4]  Anita Ganpati,et al.  A Comparative Study of Maintainability Index of Open Source Software , 2012 .

[5]  Richard Mark Soley,et al.  Software Quality Assurance: In Large Scale and Complex Software-intensive Systems , 2015 .

[6]  Panagiotis Louridas,et al.  Static code analysis , 2006, IEEE Software.

[7]  Marouane Kessentini,et al.  An introduction to modern software quality assurance , 2016 .

[8]  Nenad Medvidovic,et al.  Detecting event anomalies in event-based systems , 2015, ESEC/SIGSOFT FSE.

[9]  Nenad Medvidovic,et al.  A SEALANT for Inter-App Security Holes in Android , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE).

[10]  Jeffrey S. Foster,et al.  A comparison of bug finding tools for Java , 2004, 15th International Symposium on Software Reliability Engineering.

[11]  David Hovemeyer,et al.  Tracking defect warnings across versions , 2006, MSR '06.

[12]  Laurie Hendren,et al.  Soot: a Java bytecode optimization framework , 2010, CASCON.

[13]  William G. J. Halfond,et al.  Using Visual Symptoms for Debugging Presentation Failures in Web Applications , 2016, 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST).

[14]  Harald C. Gall,et al.  Analysing Software Repositories to Understand Software Evolution , 2008, Software Evolution.

[15]  Nenad Medvidovic,et al.  Relating Architectural Decay and Sustainability of Software Systems , 2016, 2016 13th Working IEEE/IFIP Conference on Software Architecture (WICSA).

[16]  Nenad Medvidovic,et al.  Automated Extraction of Rich Software Models from Limited System Information , 2016, 2016 13th Working IEEE/IFIP Conference on Software Architecture (WICSA).

[17]  Barry W. Boehm,et al.  Quantitative evaluation of software quality , 1976, ICSE '76.

[18]  Michael W. Godfrey,et al.  Evolution in open source software: a case study , 2000, Proceedings 2000 International Conference on Software Maintenance.

[19]  Thomas Ball,et al.  Static analysis tools as early indicators of pre-release defect density , 2005, ICSE.

[20]  Jan Jürjens,et al.  Comparing Bug Finding Tools with Reviews and Tests , 2005, TestCom.

[21]  Nenad Medvidovic,et al.  An Empirical Study of Architectural Change in Open-Source Software Systems , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[22]  Georgios Gousios,et al.  Lean GHTorrent: GitHub data on demand , 2014, MSR 2014.

[23]  Dawson R. Engler,et al.  A few billion lines of code later , 2010, Commun. ACM.

[24]  Nenad Medvidovic,et al.  A large-scale study of architectural evolution in open-source software systems , 2017, Empirical Software Engineering.