Open source vs. closed source software: towards measuring security

The increasing availability and deployment of open source software in personal and commercial environments makes open source software highly appealing for hackers, and others who are interested in exploiting software vulnerabilities. This deployment has resulted in a debate "full of religion" on the security of open source software compared to that of closed source software. However, beyond such arguments, only little quantitative analysis on this research issue has taken place. We discuss the state-of-the-art of the security debate and identify shortcomings. Based on these, we propose new metrics, which allows to answer the question to what extent the review process of open source and closed source development has helped to fix vulnerabilities. We illustrate the application of some of these metrics in a case study on OpenOffice (open source software) vs. Microsoft Office (closed source software).

[1]  Eric Rescorla,et al.  Is finding security holes a good idea? , 2005, IEEE Security & Privacy.

[2]  Indrajit Ray,et al.  Security Vulnerabilities in Software Systems: A Quantitative Perspective , 2005, DBSec.

[3]  Richard Ford.  Open vs. Closed , 2007, ACM Queue.

[4]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[5]  Christian Payne,et al.  On the security of open source software , 2002, Inf. Syst. J..

[6]  Ernesto Damiani,et al.  Adopting Open Source for Mission-Critical Applications: A Case Study on Single Sign-On , 2006, OSS.

[7]  Erland Jonsson,et al.  On the functional relation between security and dependability impairments , 1999, NSPW '99.

[8]  Mitsuhiro Kimura.  Software vulnerability: Definition, modelling, and practical evaluation for e-mail transfer software , 2006 .

[9]  Dmitri Nizovtsev,et al.  To Disclose or Not? An Analysis of Software User Behavior , 2006 .

[10]  David Wright,et al.  Towards Operational Measures of Computer Security , 1993, J. Comput. Secur..

[11]  Carl E. Landwehr,et al.  Does Open Source Improve System Security? , 2001, IEEE Softw..

[12]  Barry W. Boehm,et al.  Value Driven Security Threat Modeling Based on Attack Path Analysis , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[13]  H. Kopetz,et al.  Dependability: Basic Concepts and Terminology , 1992, Dependable Computing and Fault-Tolerant Systems.

[14]  Ken Thompson,et al.  Reflections on trusting trust , 1984, CACM.

[15]  Tomas Olovsson,et al.  On the Integration of Security and Dependability in Computer Systems , 1992 .

[16]  K Okumoto,et al.  TIME-DEPENDENT ERROR-DETECTION RATE MODEL FOR SOFTWARE AND OTHER PERFORMANCE MEASURES , 1979 .

[17]  Tomas Olovsson,et al.  A Quantitative Model of the Security Intrusion Process Based on Attacker Behavior , 1997, IEEE Trans. Software Eng..

[18]  Amrit L. Goel,et al.  Time-Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures , 1979, IEEE Transactions on Reliability.

[19]  Steven M. Bellovin.  On the Brittleness of Software and the Infeasibility of Security Metrics , 2006, IEEE Security & Privacy Magazine.

[20]  Omar H. Alhazmi,et al.  Quantitative vulnerability assessment of systems software , 2005, Annual Reliability and Maintainability Symposium, 2005. Proceedings..

[21]  Indrajit Ray,et al.  Measuring, analyzing and predicting security vulnerabilities in software systems , 2007, Comput. Secur..

[22]  Dmitri Nizovtsev,et al.  To Disclose or Not? An Analysis of Software User Behavior , 2006, Inf. Econ. Policy.

[23]  Yuanyuan Zhou,et al.  Have things changed now?: an empirical study of bug characteristics in modern open source software , 2006, ASID '06.

[24]  Robert L. Glass,et al.  A look at the economics of open source , 2004, CACM.