Increasing Software Security through Open Source or Closed Source Development? Empirics Suggest that We have Asked the Wrong Question

While many theoretical arguments against or in favor of open source and closed source software development have been presented, the empirical basis for the assessment of arguments and the development of models is still weak. Addressing this research gap, this paper presents the first comprehensive empirical investigation of published vulnerabilities and patches of 17 widely deployed open source and closed source software packages, including operating systems, database systems, web browsers, email clients, and office systems. The empirical analysis uses comprehensive vulnerability data contained in the NIST National Vulnerability Database and a newly compiled data set of vulnerability patches. The results suggest that it is not the particular software development style that determines the severity of vulnerabilities and vendors' patching behavior, but rather the specific application type and the policy of the particular development community, respectively.

[1]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[2]  William A. Arbaugh,et al.  IEEE 52 Computer , 1985 .

[3]  Jesus M. Gonzalez-Barahona.  Free Software / Open Source: Information Society Opportunities for Europe? , 2000 .

[4]  Ross J. Anderson.  Why information security is hard - an economic perspective , 2001, Seventeenth Annual Computer Security Applications Conference.

[5]  Crispin Cowan,et al.  Timing the Application of Security Patches for Optimal Uptime , 2002, LISA.

[6]  Christian Payne,et al.  On the security of open source software , 2002, Inf. Syst. J..

[7]  A. Arora,et al.  Impact of Vulnerability Disclosure and Patch Availability - An Empirical Analysis , 2004 .

[8]  Paul Kavanagh,et al.  The Open Source Definition , 2004 .

[9]  R. Anderson.  Open and Closed Systems Are Equivalent (that Is, in an Ideal World) , 2004 .

[10]  Andy Ozment,et al.  The Likelihood of Vulnerability Rediscovery and the Social Utility of Vulnerability Hunting , 2005, WEIS.

[11]  Eric Rescorla,et al.  Is finding security holes a good idea? , 2005, IEEE Security & Privacy.

[12]  Eugene H. Spafford,et al.  A Trend Analysis of Vulnerabilities , 2005 .

[13]  Rahul Telang,et al.  Competitive and Strategic Effects in the Timing of Patch Release , 2006, WEIS.

[14]  Yashwant K. Malaiya,et al.  AN ANALYSIS OF THE VULNERABILITY DISCOVERY PROCESS IN WEB BROWSERS , 2006 .

[15]  Yashwant K. Malaiya,et al.  Measuring and Enhancing Prediction Capabilities of Vulnerability Discovery Models for Apache and IIS HTTP Servers , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[16]  Jun Zhang,et al.  Economics of Security Patch Management , 2006, WEIS.

[17]  Bernhard Plattner,et al.  Large-scale vulnerability analysis , 2006, LSAD '06.

[18]  Yashwant K. Malaiya,et al.  Assessing Vulnerabilities in Apache and IIS HTTP Servers , 2006, 2006 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing.

[19]  Brian Fitzgerald,et al.  Open and Closed Systems Are Equivalent (That Is, in an Ideal World) , 2007 .

[20]  Indrajit Ray,et al.  Measuring, analyzing and predicting security vulnerabilities in software systems , 2007, Comput. Secur..

[21]  Chaim Fershtman,et al.  Network Security: Vulnerabilities and Disclosure Policy , 2007, WEIS.

[22]  Tyler Moore,et al.  Information Security Economics - and Beyond , 2007, DEON.

[23]  Dmitri Nizovtsev,et al.  To Disclose or Not? An Analysis of Software User Behavior , 2006, Inf. Econ. Policy.

[24]  Andreas Zeller,et al.  Predicting vulnerable software components , 2007, CCS '07.

[25]  Andy Ozment,et al.  Improving vulnerability discovery models , 2007, QoP '07.

[26]  Andy Ozment,et al.  Improving Vulnerability Discovery Models Problems with De fi nitions and Assumptions , 2007 .

[27]  Hao Xu,et al.  Optimal Policy for Software Vulnerability Disclosure , 2008, Manag. Sci..

[28]  Guido Schryen,et al.  Open source vs. closed source software: towards measuring security , 2009, SAC '09.

[29]  Michael Schwarz,et al.  Half a Century of Public Software Institutions: Open Source as a Solution to Hold-Up Problem , 2009 .

[30]  Guido Schryen,et al.  Security of Open Source and Closed Source Software: An Empirical Comparison of Published Vulnerabilities , 2009, AMCIS.