A comprehensive study on security bug characteristics

Security bugs can catastrophically impact our increasingly digital lives. Designing effective tools for detecting and fixing software security bugs requires a deep understanding of security bug characteristics. In this paper, we conducted a comprehensive study on security bugs and proposed the classification criteria for security bug category, that is, root cause, consequence, and location. In addition, we selected 1076 bug reports from five projects (i.e., Apache Tomcat, Apache HTTP Server, Mozilla Firefox, Linux Kernel, and Eclipse) in the NVD for investigation. Finally, we investigated the correlation between the classification results and obtained some findings: (1) memory operation is the most common security bug; (2) the primary root causes of security bugs are CON (Configuration Error), INP (Input Validation Error), and MEM (Memory Error); (3) the severity of more than 40% of security bugs is high; (4) security bugs caused by INP mainly occur on web; and (5) security bugs caused by LOG (Logic Resource Error) usually lead to DoS (Denial of Service). We discussed these findings through data analysis, which can also help developers better understand the characteristics of security bugs.

[1]  Amit M. Paradkar,et al.  A software flaw taxonomy: aiming tools at security , 2005, SOEN.

[2]  Zhengzi Xu.  Source Code and Binary Level Vulnerability Detection and Hot Patching , 2020, 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[3]  Prashant S. Shinde,et al.  Cyber security analysis using vulnerability assessment and penetration testing , 2016, 2016 World Conference on Futuristic Trends in Research and Innovation for Social Welfare (Startup Conclave).

[4]  Lutz Lowis,et al.  On a Classification Approach for SOA Vulnerabilities , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[5]  Foutse Khomh,et al.  An empirical study of patch uplift in rapid release development pipelines , 2018, Empirical Software Engineering.

[6]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[7]  Yuanyuan Zhou,et al.  Bug characteristics in open source software , 2013, Empirical Software Engineering.

[8]  F. Tsui,et al.  SOFTWARE SECURITY VULNERABILITY VS SOFTWARE COUPLING A STUDY WITH EMPIRICAL EVIDENCE A Thesis Presented to The School of Computing and Software Engineering by Varadachari Sudan Ayanam , 2009 .

[9]  Kai Zhang,et al.  How security bugs are fixed and what can be improved: an empirical study with Mozilla , 2018, Science China Information Sciences.

[10]  Ahmed E. Hassan,et al.  Security versus performance bugs: a case study on Firefox , 2011, MSR '11.

[11]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[12]  Peng Ning,et al.  Automatic diagnosis and response to memory corruption vulnerabilities , 2005, CCS '05.

[13]  Frank Piessens,et al.  A taxonomy of causes of software vulnerabilities in Internet software , 2002 .

[14]  Matthijs J. Warrens,et al.  Cohen's kappa can always be increased and decreased by combining categories , 2010 .

[15]  Nathalie Weiler,et al.  Honeypots for distributed denial-of-service attacks , 2002, Proceedings. Eleventh IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises.

[16]  S. C. Jain,et al.  Analysis and classification of SQL injection vulnerabilities and attacks on web applications , 2014, 2014 International Conference on Advances in Engineering & Technology Research (ICAETR - 2014).

[17]  Amit M. Paradkar,et al.  A software flaw taxonomy: aiming tools at security , 2005, SESS@ICSE.

[18]  A. Cantor,et al.  Sample-size calculations for Cohen's kappa. , 1996 .

[19]  Kishor S. Trivedi,et al.  A novel approach for software vulnerability classification , 2017, 2017 Annual Reliability and Maintainability Symposium (RAMS).

[20]  Rocco Oliveto,et al.  Fixing of Security Vulnerabilities in Open Source Projects: A Case Study of Apache HTTP Server and Apache Tomcat , 2019, 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST).

[21]  Andy Zaidman,et al.  Not all bugs are the same: Understanding, characterizing, and classifying bug types , 2019, J. Syst. Softw..

[22]  Yashwant K. Malaiya,et al.  Security vulnerability categories in major software systems , 2006, Communication, Network, and Information Security.