Can the Common Vulnerability Scoring System be Trusted? A Bayesian Analysis

The Common Vulnerability Scoring System (CVSS) is the state-of-the art system for assessing software vulnerabilities. However, it has been criticized for lack of validity and practitioner relevance. In this paper, the credibility of the CVSS scoring data found in five leading databases-NVD, X-Force, OSVDB, CERT-VN, and Cisco-is assessed. A Bayesian method is used to infer the most probable true values underlying the imperfect assessments of the databases, thus circumventing the problem that ground truth is not known. It is concluded that with the exception of a few dimensions, the CVSS is quite trustworthy. The databases are relatively consistent, but some are better than others. The expected accuracy of each database for a given dimension can be found by marginalizing confusion matrices. By this measure, NVD is the best and OSVDB is the worst of the assessed databases.

[1]  Laurent Gallon.  On the Impact of Environmental Metrics on CVSS Scores , 2010, 2010 IEEE Second International Conference on Social Computing.

[2]  Karen Scarfone,et al.  An analysis of CVSS version 2 vulnerability scoring , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[3]  Thomas D. Sandry,et al.  Introductory Statistics With R , 2003, Technometrics.

[4]  Stephen Wolfram,et al.  The Mathematica Book , 1996 .

[5]  Jie Tian,et al.  Text Clustering on National Vulnerability Database , 2010, 2010 Second International Conference on Computer Engineering and Applications.

[6]  A. Brix.  Bayesian Data Analysis, 2nd edn , 2005 .

[7]  Mehran Bozorgi,et al.  Beyond heuristics: learning to classify vulnerabilities and predict exploits , 2010, KDD.

[8]  Mathias Ekstedt,et al.  Empirical Analysis of System-Level Vulnerability Metrics through Actual Attacks , 2012, IEEE Transactions on Dependable and Secure Computing.

[9]  Fabio Massacci,et al.  A preliminary analysis of vulnerability scores for attacks in wild: the ekits and sym datasets , 2012, BADGERS@CCS.

[10]  Yuqing Zhang,et al.  VRSS: A new system for rating and scoring vulnerabilities , 2011, Comput. Commun..

[11]  Doina Caragea,et al.  An Empirical Study on Using the National Vulnerability Database to Predict Software Vulnerabilities , 2011, DEXA.

[12]  Fabio Massacci,et al.  Quantitative Assessment of Risk Reduction with Cybercrime Black Market Monitoring , 2013, 2013 IEEE Security and Privacy Workshops.

[13]  Ehab Al-Shaer,et al.  Vulnerability analysis For evaluating quality of protection of security policies , 2006, QoP '06.

[14]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[15]  Doina Caragea,et al.  Predicting Cyber Risks through National Vulnerability Database , 2015, Inf. Secur. J. A Glob. Perspect..

[16]  Siv Hilde Houmb,et al.  Quantifying security risk level from CVSS estimates of frequency and impact , 2010, J. Syst. Softw..

[17]  Hannes Holm,et al.  An expert-based investigation of the Common Vulnerability Scoring System , 2015, Comput. Secur..