Fixing vulnerabilities potentially hinders maintainability

Security is a requirement of utmost importance to produce highquality software. However, there is still a considerable amount of vulnerabilities being discovered and fixed almost weekly. We hypothesize that developers affect the maintainability of their codebases when patching vulnerabilities. This paper evaluates the impact of patches to improve security on the maintainability of open-source software. Maintainability is measured based on the Better Code Hub’s model of 10 guidelines on a dataset, including 1300 security-related commits. Results show evidence of a trade-off between security and maintainability for 41.90% of the cases, i.e., developers may hinder software maintainability. Our analysis shows that 38.29% of patches increased software complexity and 37.87% of patches increased the percentage of LOCs per unit. The implications of our study are that changes to codebases while patching vulnerabilities need to be performed with extra care; tools for patch risk assessment should be integrate into the CI/CD pipeline; computer science curricula needs to be updated; and, more secure programming languages are necessary.

[1]  Robert Stokes.  Beyond Fear: Thinking Sensibly about Security in an Uncertain World , 2005 .

[2]  Bruce Schneier,et al.  Beyond fear - thinking sensibly about security in an uncertain world , 2003 .

[3]  Chanchal K. Roy,et al.  Analyzing and Forecasting Near-Miss Clones in Evolving Software: An Empirical Study , 2011, 2011 16th IEEE International Conference on Engineering of Complex Computer Systems.

[4]  Jonathan Aldrich,et al.  Wyvern: Impacting Software Security via Programming Language Design , 2014, PLATEAU.

[5]  Minhaz Fahim Zibran,et al.  A Comparative Study on Vulnerabilities in Categories of Clones and Non-cloned Code , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[6]  Mayuram S. Krishnan,et al.  Evaluating the cost of software quality , 1998, CACM.

[7]  Katsuhisa Maruyama,et al.  Security-Aware Refactoring Alerting its Impact on Code Vulnerabilities , 2008, 2008 15th Asia-Pacific Software Engineering Conference.

[8]  Michelle L. Mazurek,et al.  Developers Need Support, Too: A Survey of Security Advice for Software Developers , 2017, 2017 IEEE Cybersecurity Development (SecDev).

[9]  Gabriele Bavota,et al.  On the diffuseness and the impact on maintainability of code smells: a large scale empirical investigation , 2018, Empirical Software Engineering.

[10]  Abdulrahman Abu Elkhail,et al.  On Relating Code Smells to Security Vulnerabilities , 2019, 2019 IEEE 5th Intl Conference on Big Data Security on Cloud (BigDataSecurity), IEEE Intl Conference on High Performance and Smart Computing, (HPSC) and IEEE Intl Conference on Intelligent Data and Security (IDS).

[11]  Laurie A. Williams,et al.  Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities , 2011, IEEE Transactions on Software Engineering.

[12]  Joost Visser,et al.  Standardized code quality benchmarking for improving software maintainability , 2011, Software Quality Journal.

[13]  Tibor Gyimóthy,et al.  Empirical evaluation of software maintainability based on a manually validated refactoring dataset , 2018, Inf. Softw. Technol..

[14]  A large-scale study of programming languages and code quality in GitHub , 2017 .

[15]  Rui Abreu,et al.  A Database of Existing Vulnerabilities to Enable Controlled Testing Studies , 2017, Int. J. Secur. Softw. Eng..

[16]  Jan Vitek,et al.  On the Impact of Programming Languages on Code Quality , 2019, ACM Trans. Program. Lang. Syst..

[17]  Tibor Gyimóthy,et al.  Myth or Reality? Analyzing the Effect of Design Patterns on Software Maintainability , 2012 .

[18]  Takeo Imai,et al.  A quantitative evaluation of maintainability enhancement by refactoring , 2002, International Conference on Software Maintenance, 2002. Proceedings..

[19]  Joost Visser,et al.  Benchmark-Based Aggregation of Metrics to Ratings , 2011, 2011 Joint Conference of the 21st International Workshop on Software Measurement and the 6th International Conference on Software Process and Product Measurement.

[20]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[21]  J. Pratt.  Remarks on Zeros and Ties in the Wilcoxon Signed Rank Procedures , 1959 .

[22]  Mohammad Zulkernine,et al.  Can complexity, coupling, and cohesion metrics be used as early indicators of vulnerabilities? , 2010, SAC '10.

[23]  Joost Visser,et al.  Faster issue resolution with higher technical quality of software , 2011, Software Quality Journal.

[24]  Jeroen Heijmans,et al.  A Practical Model for Rating Software Security , 2013, 2013 IEEE Seventh International Conference on Software Security and Reliability Companion.

[25]  Venkat Pothamsetty.  Where security education is lacking , 2005, InfoSecCD '05.

[26]  Rui Abreu,et al.  SECBENCH: A Database of Real Security Vulnerabilities , 2017, SecSE@ESORICS.

[27]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[28]  Jan Vitek,et al.  FSE/CACM Rebuttal2: Correcting A Large-Scale Study of Programming Languages and Code Quality in GitHub , 2019, ArXiv.

[29]  Arie van Deursen,et al.  The Delta Maintainability Model: Measuring Maintainability of Fine-Grained Code Changes , 2019, 2019 IEEE/ACM International Conference on Technical Debt (TechDebt).

[30]  Michele Bezzi,et al.  A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software , 2019, 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR).

[31]  Michael D. Ernst,et al.  Are mutants a valid substitute for real faults in software testing? , 2014, SIGSOFT FSE.

[32]  Joost Visser,et al.  A Practical Model for Measuring Maintainability , 2007, 6th International Conference on the Quality of Information and Communications Technology (QUATIC 2007).

[33]  Benjamin Chung,et al.  Wyvern: a simple, typed, and pure object-oriented language , 2013 .

[34]  Vern Paxson,et al.  A Large-Scale Empirical Study of Security Patches , 2017, CCS.

[35]  Gary Mcgraw.  Software security , 2004, IEEE Security & Privacy Magazine.

[36]  Patricia Lago,et al.  How Maintainability Issues of Android Apps Evolve , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[37]  K. McGraw,et al.  A common language effect size statistic. , 1992 .

[38]  Tiago L. Alves,et al.  Deriving metric thresholds from benchmark data , 2010, 2010 IEEE International Conference on Software Maintenance.

[39]  Foutse Khomh,et al.  Do Design Patterns Impact Software Quality Positively? , 2008, 2008 12th European Conference on Software Maintenance and Reengineering.

[40]  Forrest Shull,et al.  Investigating the impact of design debt on software quality , 2011, MTD '11.

[41]  Rahul Telang,et al.  An Empirical Analysis of the Impact of Software Vulnerability Announcements on Firm Stock Price , 2007, IEEE Transactions on Software Engineering.

[42]  Markku Sakkinen.  Proceedings of the 5th Workshop on MechAnisms for SPEcialization, Generalization and inHerItance , 2013, ECOOP 2013.

[43]  Johannes Sametinger,et al.  Software Security , 2013, 2013 20th IEEE International Conference and Workshops on Engineering of Computer Based Systems (ECBS).

[44]  Li Li,et al.  Do Energy-Oriented Changes Hinder Maintainability? , 2019, 2019 IEEE International Conference on Software Maintenance and Evolution (ICSME).