Multivariate models using MCMCBayes for web-browser vulnerability discovery

Abstract Vulnerabilities that enable well-known exploit techniques are preventable, but their public discovery continues in software. Vulnerability discovery modeling (VDM) techniques were proposed to assist managers with decisions, but do not include influential variables describing the software release (SR) (e.g., code size and complexity characteristics) and security assessment profile (SAP) (e.g., security team size or skill). Consequently, they have been limited to modeling discoveries over time for SR and SAP scenarios of unique products, whose results are not readily comparable without making assumptions that equate all SR and SAP combinations under study. This article introduces a groundbreaking capability that allows forecasting expected discoveries over time for arbitrary SR and SAP combinations, thus enabling managers to better understand the effects of influential variables they control on the phenomenon. To do this, we use variables that describe arbitrary SR and SAP combinations and construct VDM extensions that parametrically scale results from a defined baseline SR and SAP to the arbitrary SR and SAP of interest. Scaling parameters are estimated using expert judgment data gathered with a novel pairwise comparison approach. These data are then used to demonstrate predictions and how multivariate VDM techniques could be used by software-makers.

[1]  Indrajit Ray,et al.  Measuring, analyzing and predicting security vulnerabilities in software systems , 2007, Comput. Secur..

[2]  Adrian E. Raftery,et al.  Bayesian model averaging: a tutorial (with comments by M. Clyde, David Draper and E. I. George, and a rejoinder by the authors , 1999 .

[3]  Shigeru Yamada,et al.  S-Shaped Reliability Growth Modeling for Software Error Detection , 1983, IEEE Transactions on Reliability.

[4]  L. A. Goodman On the Exact Variance of Products , 1960 .

[5]  Michael P. Clements,et al.  On the limitations of comparing mean square forecast errors , 1993 .

[6]  Fabio Massacci,et al.  An Empirical Methodology to Evaluate Vulnerability Discovery Models , 2014, IEEE Transactions on Software Engineering.

[7]  Eric Rescorla,et al.  Is finding security holes a good idea? , 2005, IEEE Security & Privacy.

[8]  Jason R. W. Merrick,et al.  A Bayesian paired comparison approach for relative accident probability assessment with covariate information , 2006, Eur. J. Oper. Res..

[9]  A. Goldberger,et al.  On the Exact Covariance of Products of Random Variables , 1969 .

[10]  Tadashi Dohi,et al.  Quantitative Security Evaluation for Software System from Vulnerability Database , 2013 .

[11]  A. Pettitt,et al.  Marginal likelihood estimation via power posteriors , 2008 .

[12]  C. Robert,et al.  Bayesian Modeling Using WinBUGS , 2009 .

[13]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[14]  John D. Musa,et al.  Software reliability - measurement, prediction, application , 1987, McGraw-Hill series in software engineering and technology.

[15]  Yashwant K. Malaiya,et al.  Modeling vulnerability discovery process in Apache and IIS HTTP servers , 2011, Comput. Secur..

[16]  Eldad Eilam,et al.  Reversing: Secrets of Reverse Engineering , 2005 .

[17]  Christian S. Collberg,et al.  Watermarking, Tamper-Proofing, and Obfuscation-Tools for Software Protection , 2002, IEEE Trans. Software Eng..

[18]  Andy Ozment,et al.  Software Security Growth Modeling: Examining Vulnerabilities with Reliability Growth Models , 2006, Quality of Protection.

[19]  Mehdi R. Zargham,et al.  Vulnerability Scrying Method for Software Vulnerability Discovery Prediction Without a Vulnerability Database , 2013, IEEE Transactions on Reliability.

[20]  Ville Leppänen,et al.  The sigmoidal growth of operating system security vulnerabilities: An empirical revisit , 2015, Comput. Secur..

[21]  Michael R. Lyu,et al.  An Assessment of Testing-Effort Dependent Software Reliability Growth Models , 2007, IEEE Transactions on Reliability.

[22]  Yashwant K. Malaiya,et al.  Application of Vulnerability Discovery Models to Major Operating Systems , 2008, IEEE Transactions on Reliability.

[23]  Daniel J. Ryan,et al.  Quantifying information security risks using expert judgment elicitation , 2012, Comput. Oper. Res..

[24]  Thomas Ball,et al.  The concept of dynamic analysis , 1999, ESEC/FSE-7.

[25]  Refik Soyer,et al.  Modeling and Analysis of Call Center Arrival Data: A Bayesian Approach , 2008, Manag. Sci..

[26]  Shigeru Yamada,et al.  TESTING-DOMAIN DEPENDENT SOFTWARE RELIABILITY GROWTH MODELS AND THEIR COMPARISONS OF GOODNESS-OF-FIT , 2001 .

[27]  Tiago M. Fragoso,et al.  Bayesian Model Averaging: A Systematic Review and Conceptual Classification , 2015, 1509.08864.

[28]  Omar H. Alhazmi,et al.  Quantitative vulnerability assessment of systems software , 2005, Annual Reliability and Maintainability Symposium, 2005. Proceedings..

[29]  R. Cooke Experts in Uncertainty: Opinion and Subjective Probability in Science , 1991 .

[30]  G. C. Tiao,et al.  Bayesian inference in statistical analysis , 1973 .

[31]  Mitsuhiro Kimura Software vulnerability: Definition, modelling, and practical evaluation for e-mail transfer software , 2006 .

[32]  John D. Musa,et al.  A theory of software reliability and its application , 1975, IEEE Transactions on Software Engineering.

[33]  Andy Ozment,et al.  Improving vulnerability discovery models , 2007, QoP '07.

[34]  Yaman Roumani,et al.  Time series modeling of vulnerabilities , 2015, Comput. Secur..

[35]  R. A. Bradley,et al.  RANK ANALYSIS OF INCOMPLETE BLOCK DESIGNS , 1952 .

[36]  Yashwant K. Malaiya,et al.  Modeling Skewness in Vulnerability Discovery , 2014, Qual. Reliab. Eng. Int..

[37]  Bev Littlewood How to Measure Software Reliability and How Not To , 1979, IEEE Transactions on Reliability.