Investigating the vulnerability fixing process in OSS projects: Peculiarities and challenges

Abstract Although vulnerabilities can be considered and treated as bugs, they present numerous peculiarities compared to other types of bugs (canonical bugs in the remainder of the paper). A vulnerability adds functionality to a system, as it allows an adversary to misuse or abuse the system, while a canonical bug is an incomplete or incorrect implementation of a requirement, and thus degrades the functionality of the system. This difference can affect the fixing process of vulnerabilities. By mining the repositories of 6 open source projects, we characterize the differences in the fixing process between vulnerabilities and canonical bugs, highlighting critical issues which could represent challenges for future research. Results of our study demonstrate that: (i) more re-assignments (than the ones observed in canonical bugs) are required for finding the developers able to handle vulnerability-related bugs, (ii) developers’ security-related skills should be profiled, to improve the efficiency of the security bug assignment tasks, and, consequently, reduce the re-assignments, and (iii) vulnerabilities require more effort, contributors and time to define the fixing strategy but smaller time to fix than canonical bugs.

[1]  Rocco Oliveto,et al.  Fixing of Security Vulnerabilities in Open Source Projects: A Case Study of Apache HTTP Server and Apache Tomcat , 2019, 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST).

[2]  Lucas D. Panjer Predicting Eclipse Bug Lifetimes , 2007, Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007).

[3]  Chao Liu,et al.  An Approach to Improving Bug Assignment with Bug Tossing Graphs and Bug Similarities , 2011, J. Softw..

[4]  Achim D. Brucker,et al.  Time for Addressing Software Security Issues: Prediction Models and Impacting Factors , 2017, Data Science and Engineering.

[5]  Baldoino Fonseca dos Santos Neto,et al.  Software Metrics and Security Vulnerabilities: Dataset and Exploratory Study , 2016, 2016 12th European Dependable Computing Conference (EDCC).

[6]  Liudmila Ulanova,et al.  An Empirical Analysis of Bug Reports and Bug Fixing in Open Source Android Apps , 2013, 2013 17th European Conference on Software Maintenance and Reengineering.

[7]  Andrew Meneely,et al.  Do Bugs Foreshadow Vulnerabilities? A Study of the Chromium Project , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[8]  Thomas Zimmermann,et al.  Improving bug triage with bug tossing graphs , 2009, ESEC/FSE '09.

[9]  Tao Xie,et al.  Identifying security bug reports via text mining: An industrial case study , 2010, 2010 7th IEEE Working Conference on Mining Software Repositories (MSR 2010).

[10]  Zhendong Su,et al.  An Empirical Study on Real Bug Fixes , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[11]  Yongjian Fu,et al.  A Case Study on Design Patterns and Software Defects in Open Source Software , 2018 .

[12]  Nuno Laranjeiro,et al.  An Analysis of OpenStack Vulnerabilities , 2017, 2017 13th European Dependable Computing Conference (EDCC).

[13]  Achim D. Brucker,et al.  Factors Impacting the Effort Required to Fix Security Vulnerabilities - An Industrial Case Study , 2015, ISC.

[14]  Miryung Kim,et al.  An empirical study of supplementary bug fixes , 2012, 2012 9th IEEE Working Conference on Mining Software Repositories (MSR).

[15]  Gerardo Canfora,et al.  Supporting change request assignment in open source development , 2006, SAC.

[16]  Gail C. Murphy,et al.  Who should fix this bug? , 2006, ICSE.

[17]  Aniello Cimitile,et al.  An exploratory study on the evolution of Android malware quality , 2018, J. Softw. Evol. Process..

[18]  Zhenchang Xing,et al.  Characterizing Common and Domain-Specific Package Bugs: A Case Study on Ubuntu , 2018, 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC).

[19]  Diomidis Spinellis,et al.  An Empirical Analysis of Vulnerabilities in Virtualization Technologies , 2016, 2016 IEEE International Conference on Cloud Computing Technology and Science (CloudCom).

[20]  Ahmed E. Hassan,et al.  Security versus performance bugs: a case study on Firefox , 2011, MSR '11.

[21]  Foutse Khomh,et al.  An Empirical Study on Factors Impacting Bug Fixing Time , 2012, 2012 19th Working Conference on Reverse Engineering.

[22]  David Lo,et al.  Which Packages Would be Affected by This Bug Report? , 2017, 2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE).

[23]  W. Pirie Spearman Rank Correlation Coefficient , 2006 .

[24]  W. J. Conover,et al.  Practical Nonparametric Statistics , 1972 .

[25]  Harald C. Gall,et al.  Predicting the fix time of bugs , 2010, RSSE '10.

[26]  Roberto Almeida Bittencourt,et al.  Do Rapid Releases Affect Bug Reopening? A Case Study of Firefox , 2014, 2014 Brazilian Symposium on Software Engineering.

[27]  Bo Zhou,et al.  A cross-platform analysis of bugs and bug-fixing in open source projects: desktop vs. Android vs. iOS , 2015, EASE.

[28]  Sunghun Kim,et al.  How long did it take to fix bugs? , 2006, MSR '06.

[29]  HyunChul Joh Quantitative security analysis of network OSes by fitting VDM and examining CVSS , 2018, 2018 International Conference on Information Networking (ICOIN).

[30]  Gerardo Canfora,et al.  Summarizing vulnerabilities' descriptions to support experts during vulnerability assessment activities , 2019, J. Syst. Softw..

[31]  Andreas Zeller,et al.  When do changes induce fixes? , 2005, ACM SIGSOFT Softw. Eng. Notes.

[32]  R. Grissom,et al.  Effect sizes for research: A broad practical approach. , 2005 .

[33]  Andreas Zeller,et al.  How Long Will It Take to Fix This Bug? , 2007, Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007).

[34]  Laurie A. Williams,et al.  Are vulnerabilities discovered and resolved like other defects? , 2017, Empirical Software Engineering.

[35]  David Lo,et al.  A Deeper Look into Bug Fixes: Patterns, Replacements, Deletions, and Additions , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[36]  Andy Zaidman,et al.  Not all bugs are the same: Understanding, characterizing, and classifying bug types , 2019, J. Syst. Softw..

[37]  Hong Mei,et al.  A survey on bug-report analysis , 2015, Science China Information Sciences.

[38]  Kapil Tarey,et al.  A Review on Taxonomies of Attacks and Vulnerability in Computer and Network System , 2015 .

[39]  Vern Paxson,et al.  A Large-Scale Empirical Study of Security Patches , 2017, CCS.

[40]  Yaqin Zhou,et al.  Automated identification of security issues from commit messages and bug reports , 2017, ESEC/SIGSOFT FSE.

[41]  Riccardo Scandariato,et al.  Predicting Vulnerable Components: Software Metrics vs Text Mining , 2014, 2014 IEEE 25th International Symposium on Software Reliability Engineering.