Vulnerability prediction capability: A comparison between vulnerability discovery models and neural network models

In this paper, we introduce an approach for predicting the cumulative number of software vulnerabilities that is in most cases more accurate than vulnerability discovery models (VDMs). Our approach uses a neural network model (NNM) to model the nonlinearities associated with vulnerability disclosure. Nine common VDMs were used to compare their prediction capability with our approach. The different models were applied to vulnerabilities associated with eight well-known software (four operating systems and four web browsers). The models were assessed in terms of prediction accuracy and prediction bias. Out of eight software we analyzed, the NNM outperformed the VDMs in all the cases in terms of prediction accuracy, and provided smaller values of absolute average bias in seven cases. This study shows that NNMs are promising for accurate predictions of software vulnerabilities disclosures.

[1]  Berna Yazici,et al.  Comparison of ARIMA, neural networks and hybrid models in time series: tourist arrival forecasting , 2007 .

[2]  Yashwant K. Malaiya,et al.  Application of Vulnerability Discovery Models to Major Operating Systems , 2008, IEEE Transactions on Reliability.

[3]  Amarda Shehu,et al.  User Behavior Modelling for Fake Information Mitigation on Social Web , 2019, SBP-BRiMS.

[4]  Giovanni Besio,et al.  Problems in RMSE-based wave model validations , 2013 .

[5]  Yaman Roumani,et al.  Time series modeling of vulnerabilities , 2015, Comput. Secur..

[6]  Yashwant K. Malaiya,et al.  Modeling the vulnerability discovery process , 2005, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05).

[7]  Mary Shaw,et al.  Selecting a Defect Prediction Model for Maintenance Resource Planning and Software Insurance , 2003 .

[8]  Luca Allodi,et al.  The Heavy Tails of Vulnerability Exploitation , 2015, ESSoS.

[9]  Yashwant K. Malaiya,et al.  Modeling Skewness in Vulnerability Discovery , 2014, Qual. Reliab. Eng. Int..

[10]  Indrakshi Ray,et al.  Vulnerability Discovery in Multi-Version Software Systems , 2007 .

[11]  Michael Y. Hu,et al.  Forecasting with artificial neural networks: The state of the art , 1997 .

[12]  Ilir Gashi,et al.  Cluster-Based Vulnerability Assessment Applied to Operating Systems , 2017, 2017 13th European Dependable Computing Conference (EDCC).

[13]  Holger R. Maier,et al.  Review of Input Variable Selection Methods for Artificial Neural Networks , 2011 .

[14]  Tao Chen,et al.  Back propagation neural network with adaptive differential evolution algorithm for time series forecasting , 2015, Expert Syst. Appl..

[15]  Chris P. Tsokos,et al.  Cybersecurity: Time Series Predictive Modeling of Vulnerabilities of Desktop Operating System Using Linear and Non-Linear Approach , 2017 .

[16]  P. K. Kapur,et al.  Vulnerability discovery model for a software system using stochastic differential equation , 2015, 2015 International Conference on Futuristic Trends on Computational Analysis and Knowledge Management (ABLAZE).

[17]  Eric Rescorla,et al.  Is finding security holes a good idea? , 2005, IEEE Security & Privacy.

[18]  Fabio Massacci,et al.  An automatic method for assessing the versions affected by a vulnerability , 2015, Empirical Software Engineering.

[19]  Omar H. Alhazmi,et al.  Quantitative vulnerability assessment of systems software , 2005, Annual Reliability and Maintainability Symposium, 2005. Proceedings..

[20]  Eric Rescorla Security Holes . . . Who Cares? , 2003, USENIX Security Symposium.

[21]  Rodney A. Stewart,et al.  ANN-based residential water end-use demand forecasting model , 2013, Expert Syst. Appl..

[22]  David Wright,et al.  Towards Operational Measures of Computer Security , 1993, J. Comput. Secur..

[23]  Lynn Kuo,et al.  Bayesian computation for the superposition of nonhomogeneous poisson processes , 1999 .

[24]  Yashwant K. Malaiya,et al.  Measuring and Enhancing Prediction Capabilities of Vulnerability Discovery Models for Apache and IIS HTTP Servers , 2006, 2006 17th International Symposium on Software Reliability Engineering.

[25]  Nikolaos Kourentzes,et al.  Neural network ensemble operators for time series forecasting , 2014, Expert Syst. Appl..

[26]  Yazdan Movahedi Some Guidelines for Risk Assessment of Vulnerability Discovery Processes , 2019 .

[27]  H Gholam Hosseini,et al.  The comparison of different feed forward neural network architectures for ECG signal diagnosis. , 2006, Medical engineering & physics.

[28]  Ilir Gashi,et al.  Cluster-based vulnerability assessment of operating systems and web browsers , 2018, Computing.

[29]  Aderemi Oluyinka Adewumi,et al.  Comparison of ARIMA and Artificial Neural Networks Models for Stock Price Prediction , 2014, J. Appl. Math..

[30]  Mary Shaw,et al.  Empirical evaluation of defect projection models for widely-deployed production software systems , 2004, SIGSOFT '04/FSE-12.

[31]  Fabio Massacci,et al.  An Empirical Methodology to Evaluate Vulnerability Discovery Models , 2014, IEEE Transactions on Software Engineering.