Can complexity, coupling, and cohesion metrics be used as early indicators of vulnerabilities?

It is difficult to detect vulnerabilities until they manifest themselves as security failures in the operational stage of software, because the security concerns are not addressed or known sufficiently early during software development. Complexity, coupling, and cohesion (CCC) related software metrics can be measured during the earlier phases of software development. If empirical relationships can be discovered between CCC metrics and vulnerabilities, these metrics could aid software developers to take proactive actions against potential vulnerabilities in software. In this paper, we conduct an extensive case study on Mozilla Firefox to provide empirical evidence on how vulnerabilities are related to complexity, coupling, and cohesion. We find that CCC metrics are correlated to vulnerabilities at a statistically significant level. We further examine the correlations to determine which level (design or code) of CCC metrics are better indicators of vulnerabilities. We also observe that the correlation patterns are stable across multiple releases of the software. These observations show that CCC metrics can be dependably used as early indicators of vulnerabilities in software.

[1]  Andreas Zeller,et al.  Predicting vulnerable software components , 2007, CCS '07.

[2]  Issa Traoré,et al.  Empirical relation between coupling and attackability in software systems:: a case study on DOS , 2006, PLAS '06.

[3]  Chris F. Kemerer,et al.  A Metrics Suite for Object Oriented Design , 2015, IEEE Trans. Software Eng..

[4]  Akif Günes Koru,et al.  An empirical comparison and characterization of high defect and high complexity modules , 2003, J. Syst. Softw..

[5]  Mohammad Zulkernine,et al.  Security metrics for source code structures , 2008, SESS '08.

[6]  Shari Lawrence Pfleeger,et al.  Software metrics (2nd ed.): a rigorous and practical approach , 1997 .

[7]  Shari Lawrence Pfleeger,et al.  Software Metrics : A Rigorous and Practical Approach , 1998 .

[8]  Giuliano Antoniol,et al.  Threats on building models from CVS and Bugzilla repositories: the Mozilla case study , 2007, CASCON.

[9]  Laurie A. Williams,et al.  Is complexity really the enemy of software security? , 2008, QoP '08.

[10]  Ahmed E. Hassan,et al.  Mining Software Repositories to Assist Developers and Support Managers , 2006, 2006 22nd IEEE International Conference on Software Maintenance.

[11]  Laurie A. Williams,et al.  An empirical model to predict security vulnerabilities using code complexity metrics , 2008, ESEM '08.

[12]  Jessica Keyes,et al.  IEEE Standard Dictionary of Measures to Produce Reliable Software , 2002 .

[13]  Ahmed E. Hassan,et al.  Understanding the rationale for updating a function’s comment , 2008, 2008 IEEE International Conference on Software Maintenance.

[14]  Witold Pedrycz,et al.  Identification of defect-prone classes in telecommunication software systems using design metrics , 2006, Inf. Sci..

[15]  Tim Menzies,et al.  Data Mining Static Code Attributes to Learn Defect Predictors , 2007, IEEE Transactions on Software Engineering.

[16]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[17]  Mladen A. Vouk,et al.  Towards a Metric Suite for Early Software Reliability Assessment , 2003 .

[18]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[19]  Indrajit Ray,et al.  Measuring, analyzing and predicting security vulnerabilities in software systems , 2007, Comput. Secur..

[20]  Andreas Zeller,et al.  Mining metrics to predict component failures , 2006, ICSE.

[21]  Javam C. Machado,et al.  The prediction of faulty classes using object-oriented design metrics , 2001, J. Syst. Softw..

[22]  Ming Gu,et al.  Predicting Defective Software Components from Code Complexity Measures , 2007, 13th Pacific Rim International Symposium on Dependable Computing (PRDC 2007).