Model-Based Quantitative Network Security Metrics: A Survey

Network security metrics (NSMs) based on models allow to quantitatively evaluate the overall resilience of networked systems against attacks. For that reason, such metrics are of great importance to the security-related decision-making process of organizations. Considering that over the past two decades several model-based quantitative NSMs have been proposed, this paper presents a deep survey of the state-of-the-art of these proposals. First, to distinguish the security metrics described in this survey from other types of security metrics, an overview of security metrics, in general, and their classifications is presented. Then, a detailed review of the main existing model-based quantitative NSMs is provided, along with their advantages and disadvantages. Finally, this survey is concluded with an in-depth discussion on relevant characteristics of the surveyed proposals and open research issues of the topic.

[1]  Matt Bishop,et al.  What Is Computer Security? , 2003, IEEE Secur. Priv..

[2]  Ram Dantu,et al.  Risk Management Using Behavior Based Bayesian Networks , 2005, ISI.

[3]  Steven M. Bellovin.  On the Brittleness of Software and the Infeasibility of Security Metrics , 2006, IEEE Security & Privacy Magazine.

[4]  Wang Huiqiang,et al.  Study of Network Security Situation Awareness Model Based on Simple Additive Weight and Grey Theory , 2006, 2006 International Conference on Computational Intelligence and Security.

[5]  Lingyu Wang,et al.  Measuring Network Security Using Bayesian Network-Based Attack Graphs , 2008, 2008 32nd Annual IEEE International Computer Software and Applications Conference.

[6]  Duminda Wijesekera,et al.  Scalable, graph-based network vulnerability analysis , 2002, CCS '02.

[7]  Shirley C. Payne,et al.  A Guide to Security Metrics , 2007 .

[8]  William H. Sanders,et al.  Implementing the ADVISE security modeling formalism in Möbius , 2013, 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[9]  Salvatore J. Stolfo,et al.  Measuring Security , 2011, IEEE Security & Privacy.

[10]  Barry O'Sullivan,et al.  Multilevel Security and Quality of Protection , 2006, Quality of Protection.

[11]  Galen H. Sasaki,et al.  Quality of protection (QoP): a quantitative unifying paradigm to protection service grades , 2001, OptiComm: Optical Networking and Communications Conference.

[12]  Bharat B. Madan,et al.  A method for modeling and quantifying the security attributes of intrusion tolerant systems , 2004, Perform. Evaluation.

[13]  Svein J. Knapskog,et al.  Towards a stochastic model for integrated security and dependability evaluation , 2006, First International Conference on Availability, Reliability and Security (ARES'06).

[14]  Karen Scarfone,et al.  An analysis of CVSS version 2 vulnerability scoring , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[15]  Vojkan Mihajlovic,et al.  Dynamic Bayesian Networks: A State of the Art , 2001 .

[16]  Erland Jonsson,et al.  How to systematically classify computer security intrusions , 1997, S&P 1997.

[17]  Sushil Jajodia,et al.  Toward measuring network security using attack graphs , 2007, QoP '07.

[18]  Bharat B. Madan,et al.  Modeling and quantification of security attributes of software systems , 2002, Proceedings International Conference on Dependable Systems and Networks.

[19]  Debra Herrmann,et al.  Complete Guide to Security and Privacy Metrics: Measuring Regulatory Compliance, Operational Resilience, and ROI , 2007 .

[20]  Brian Randell,et al.  Fundamental Concepts of Dependability , 2000 .

[21]  Guofei Gu,et al.  Measuring intrusion detection capability: an information-theoretic approach , 2006, ASIACCS '06.

[22]  Ioannis Mavridis,et al.  Information infrastructure risk prediction through platform vulnerability analysis , 2015, J. Syst. Softw..

[23]  P. Bhattacharya,et al.  Analytical framework for measuring network security using exploit dependency graph , 2012, IET Inf. Secur..

[24]  Somesh Jha,et al.  Two formal analyses of attack graphs , 2002, Proceedings 15th IEEE Computer Security Foundations Workshop. CSFW-15.

[25]  Nicholas Kyriakopoulos,et al.  A comparative analysis of network dependability, fault-tolerance, reliability, security, and survivability , 2009, IEEE Communications Surveys & Tutorials.

[26]  Raimir Holanda Filho,et al.  Quantifying Node Security in Wireless Sensor Networks under Worm Attacks , 2017 .

[27]  Bharat K. Bhargava,et al.  Extending Attack Graph-Based Security Metrics and Aggregating Their Application , 2012, IEEE Transactions on Dependable and Secure Computing.

[28]  Reijo Savola,et al.  Towards a Security Metrics Taxonomy for the Information and Communication Technology Industry , 2007, International Conference on Software Engineering Advances (ICSEA 2007).

[29]  Andrew W. Appel,et al.  MulVAL: A Logic-based Network Security Analyzer , 2005, USENIX Security Symposium.

[30]  Ehab Al-Shaer,et al.  Objective Risk Evaluation for Automated Security Management , 2010, Journal of Network and Systems Management.

[31]  Sushil Jajodia,et al.  Managing attack graph complexity through visual hierarchical aggregation , 2004, VizSEC/DMSEC '04.

[32]  Sushil Jajodia,et al.  Measuring Security Risk of Networks Using Attack Graphs , 2010, Int. J. Next Gener. Comput..

[33]  Wayne A. Jansen,et al.  Directions in Security Metrics Research , 2009 .

[34]  Shouhuai Xu,et al.  An Extended Stochastic Model for Quantitative Security Analysis of Networked Systems , 2012, Internet Math..

[35]  Artur Hecker,et al.  On System Security Metrics and the Definition Approaches , 2008, 2008 Second International Conference on Emerging Security Information, Systems and Technologies.

[36]  Jorge Sá Silva,et al.  Security for the Internet of Things: A Survey of Existing Protocols and Open Research Issues , 2015, IEEE Communications Surveys & Tutorials.

[37]  Paul Ammann,et al.  Using model checking to analyze network vulnerabilities , 2000, Proceeding 2000 IEEE Symposium on Security and Privacy. S&P 2000.

[38]  Teresa F. Lunt,et al.  A survey of intrusion detection techniques , 1993, Comput. Secur..

[39]  Wang Chunlei,et al.  A framework for network security situation awareness based on knowledge discovery , 2010, 2010 2nd International Conference on Computer Engineering and Technology.

[40]  Miles A. McQueen,et al.  Quantitative Cyber Risk Reduction Estimation Methodology for a Small SCADA Control System , 2006, Proceedings of the 39th Annual Hawaii International Conference on System Sciences (HICSS'06).

[41]  Karen Scarfone,et al.  Common Vulnerability Scoring System , 2006, IEEE Security & Privacy.

[42]  Mathias Ekstedt,et al.  Empirical Analysis of System-Level Vulnerability Metrics through Actual Attacks , 2012, IEEE Transactions on Dependable and Secure Computing.

[43]  Sushil Jajodia,et al.  Modeling Network Diversity for Evaluating the Robustness of Networks against Zero-Day Attacks , 2014, ESORICS.

[44]  Bogdan Ksiezopolski,et al.  QoP-ML: Quality of protection modelling language for cryptographic protocols , 2012, Comput. Secur..

[45]  Rodolphe Ortalo,et al.  Experimenting with Quantitative Evaluation Tools for Monitoring Operational Security , 1999, IEEE Trans. Software Eng..

[46]  Jennifer L. Bayuk,et al.  Security as a theoretical attribute construct , 2013, Comput. Secur..

[47]  David Wright,et al.  Towards Operational Measures of Computer Security , 1993, J. Comput. Secur..

[48]  Rainer Böhme,et al.  Economic Security Metrics , 2005, Dependability Metrics.

[49]  Saman A. Zonouz,et al.  Cyber-physical security metric inference in smart grid critical infrastructures based on system administrators' responsive behavior , 2013, Comput. Secur..

[50]  Indrajit Ray,et al.  Dynamic Security Risk Management Using Bayesian Attack Graphs , 2012, IEEE Transactions on Dependable and Secure Computing.

[51]  Mario Piattini,et al.  A comparison of software design security metrics , 2010, ECSA '10.

[52]  Subil Mathew Abraham.  Estimating Mean Time to Compromise Using Non-homogenous Continuous-Time Markov Models , 2016, 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC).

[53]  David A. Schmidt,et al.  Aggregating vulnerability metrics in enterprise networks using attack graphs , 2013, J. Comput. Secur..

[54]  Yifan Li,et al.  VisFlowConnect: netflow visualizations of link relationships for security situational awareness , 2004, VizSEC/DMSEC '04.

[55]  Ian Lumb,et al.  A Taxonomy and Survey of Cloud Computing Systems , 2009, 2009 Fifth International Joint Conference on INC, IMS and IDC.

[56]  Steven Furnell,et al.  Assessing the Feasibility of Security Metrics , 2013, TrustBus.

[57]  Kevin Jones,et al.  A review of cyber security risk assessment methods for SCADA systems , 2016, Comput. Secur..

[58]  Mario Piattini,et al.  Towards a Classification of Security Metrics , 2004, WOSIS.

[59]  Sushil Jajodia,et al.  A weakest-adversary security metric for network configuration security analysis , 2006, QoP '06.

[60]  Samir Ouchani,et al.  Specification, verification, and quantification of security in model-based systems , 2015, Computing.

[61]  Jeannette M. Wing,et al.  An Attack Surface Metric , 2011, IEEE Transactions on Software Engineering.

[62]  Jianbin Hu,et al.  Applying Attack Graphs to Network Security Metric , 2009, 2009 International Conference on Multimedia Information Networking and Security.

[63]  Leyla Bilge,et al.  Before we knew it: an empirical study of zero-day attacks in the real world , 2012, CCS.

[64]  Sushil Jajodia,et al.  Diversifying Network Services Under Cost Constraints for Better Resilience Against Unknown Attacks , 2016, DBSec.

[65]  Feiyi Wang,et al.  SITAR: a scalable intrusion-tolerant architecture for distributed services , 2003, Foundations of Intrusion Tolerant Systems, 2003 [Organically Assured and Survivable Information Systems].

[66]  Miles A. McQueen,et al.  Ideal Based Cyber Security Technical Metrics for Control Systems , 2007, CRITIS.

[67]  Ali A. Ghorbani,et al.  A Survey of Visualization Systems for Network Security , 2012, IEEE Transactions on Visualization and Computer Graphics.

[68]  William H. Sanders,et al.  Model-based evaluation: from dependability to security , 2004, IEEE Transactions on Dependable and Secure Computing.

[69]  Mehmet Sahinoglu,et al.  Security meter: a practical decision-tree model to quantify risk , 2005, IEEE Security & Privacy Magazine.

[70]  Marc Dacier,et al.  Empirical analysis and statistical modeling of attack processes based on honeypots , 2007, ArXiv.

[71]  William H. Sanders,et al.  Model-based Security Metrics Using ADversary VIew Security Evaluation (ADVISE) , 2011, 2011 Eighth International Conference on Quantitative Evaluation of SysTems.

[72]  Marc Dacier,et al.  Quantitative Assessment of Operational Security: Models and Tools * , 1996 .

[73]  Kaoru Ota,et al.  Quality of experience and quality of protection provisions in emerging mobile networks [Guest Editorial] , 2015, IEEE Wireless Communications.

[74]  Ram Dantu,et al.  Network risk management using attacker profiling , 2009, Secur. Commun. Networks.

[75]  Sushil Jajodia,et al.  k-Zero Day Safety: A Network Security Metric for Measuring the Risk of Unknown Vulnerabilities , 2014, IEEE Transactions on Dependable and Secure Computing.

[76]  William H. Sanders,et al.  Seclius: An Information Flow-Based, Consequence-Centric Security Metric , 2015, IEEE Transactions on Parallel and Distributed Systems.

[77]  Miles A. McQueen,et al.  Time-to-Compromise Model for Cyber Risk Reduction Estimation , 2006, Quality of Protection.

[78]  Marianne Swanson,et al.  Security metrics guide for information technology systems , 2003 .

[79]  Edmund M. Clarke,et al.  Ranking Attack Graphs , 2006, RAID.

[80]  Ray Hunt,et al.  A taxonomy of network and computer attacks , 2005, Comput. Secur..

[81]  N. Paulauskas,et al.  Attacker Skill Level distribution estimation in the system mean time-to-compromise , 2008, 2008 1st International Conference on Information Technology.

[82]  R. Cunningham,et al.  Validating and Restoring Defense in Depth Using Attack Graphs , 2006, MILCOM 2006 - 2006 IEEE Military Communications conference.

[83]  Reinhard Schwarz,et al.  A Critical Survey of Security Indicator Approaches , 2012, 2012 Seventh International Conference on Availability, Reliability and Security.

[84]  Rayford B. Vaughn,et al.  Information assurance measures and metrics - state of practice and proposed taxonomy , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[85]  Martin Gilje Jaatun,et al.  Hunting for Aardvarks: Can Software Security Be Measured? , 2012, CD-ARES.

[86]  Jagath Samarabandu,et al.  Security Analysis and Auditing of IEC61850-Based Automated Substations , 2010, IEEE Transactions on Power Delivery.

[87]  Mohammad Abdollahi Azgomi,et al.  Mean privacy: A metric for security of computer systems , 2014, Comput. Commun..

[88]  Rayford B. Vaughn,et al.  Cluster Security Research Involving the Modeling of Network Exploitations Using Exploitation Graphs , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[89]  Marcus Pendleton,et al.  A Survey on Systems Security Metrics , 2016, ACM Comput. Surv..

[90]  Chandan Mazumdar,et al.  Attack Graph Generation and Analysis Techniques , 2016 .

[91]  Tomas Olovsson,et al.  A Quantitative Model of the Security Intrusion Process Based on Attacker Behavior , 1997, IEEE Trans. Software Eng..

[92]  Yang Lin,et al.  Quality of Protection in Web Service: An Overview , 2011, 2011 First International Conference on Instrumentation, Measurement, Computer, Communication and Control.

[93]  Kresimir Solic,et al.  The information systems' security level assessment model based on an ontology and evidential reasoning approach , 2015, Comput. Secur..

[94]  Lawrence C. Stewart,et al.  HTTP Authentication: Basic and Digest Access Authentication , 1999 .

[95]  Sushil Jajodia,et al.  Topological analysis of network attack vulnerability , 2006, PST.

[96]  Yifan Li,et al.  VisFlowConnect: providing security situational awareness by visualizing network traffic flows , 2004, IEEE International Conference on Performance, Computing, and Communications, 2004.

[97]  Svein J. Knapskog,et al.  On Stochastic Modeling for Integrated Security and Dependability Evaluation , 2006, J. Networks.

[98]  Saman A. Zonouz,et al.  CPIndex: Cyber-Physical Vulnerability Assessment for Power-Grid Infrastructures , 2015, IEEE Transactions on Smart Grid.

[99]  Shouhuai Xu,et al.  A Stochastic Model for Quantitative Security Analyses of Networked Systems , 2016, IEEE Transactions on Dependable and Secure Computing.

[100]  David John Leversage,et al.  Estimating a System's Mean Time-to-Compromise , 2008, IEEE Security & Privacy.

[101]  Sushil Jajodia,et al.  Measuring network security using dynamic bayesian network , 2008, QoP '08.

[102]  Marc Dacier,et al.  Models and tools for quantitative assessment of operational security , 1996, SEC.

[103]  Sajjan G. Shiva,et al.  Use of Attack Graphs in Security Systems , 2014, J. Comput. Networks Commun..

[104]  Mohammad Abdollahi Azgomi,et al.  Intrusion Process Modeling for Security Quantification , 2009, 2009 International Conference on Availability, Reliability and Security.

[105]  Ehab Al-Shaer,et al.  Private and Anonymous Data Storage and Distribution in Cloud , 2013, 2013 IEEE International Conference on Services Computing.

[106]  Birgit Pfitzmann,et al.  Service-oriented Assurance - Comprehensive Security by Explicit Assurances , 2006, Quality of Protection.

[107]  Mohammad Abdollahi Azgomi,et al.  A Method for Estimation of the Success Probability of an Intrusion Process by Considering the Temporal Aspects of the Attacker Behavior , 2009, Trans. Comput. Sci..

[108]  Sushil Jajodia,et al.  Measuring the Overall Security of Network Configurations Using Attack Graphs , 2007, DBSec.

[109]  John McHugh.  Quality of protection: measuring the unmeasurable? , 2006, QoP '06.

[110]  Vilhelm Verendel,et al.  Quantified security is a weak hypothesis: a critical survey of results and assumptions , 2009, NSPW '09.

[111]  Mohsen Guizani,et al.  Internet of Things: A Survey on Enabling Technologies, Protocols, and Applications , 2015, IEEE Communications Surveys & Tutorials.

[112]  Felix C. Freiling,et al.  On Metrics and Measurements , 2005, Dependability Metrics.

[113]  Giovanni Vigna,et al.  NetSTAT: A Network-based Intrusion Detection System , 1999, J. Comput. Secur..

[114]  Ian F. Akyildiz,et al.  Sensor Networks , 2002, Encyclopedia of GIS.

[115]  David Hutchison,et al.  A survey of cyber security management in industrial control systems , 2015, Int. J. Crit. Infrastructure Prot..

[116]  Andrew Jaquith.  Security Metrics: Replacing Fear, Uncertainty, and Doubt , 2007 .

[117]  Peng Liu,et al.  Using Bayesian networks for cyber security analysis , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[118]  Raimir Holanda Filho,et al.  Sensor Data Security Level Estimation Scheme for Wireless Sensor Networks , 2015, Sensors.

[119]  Sushil Jajodia,et al.  Network Diversity: A Security Metric for Evaluating the Resilience of Networks Against Zero-Day Attacks , 2016, IEEE Transactions on Information Forensics and Security.

[120]  A. Nur Zincir-Heywood,et al.  VEA-bility Security Metric: A Network Security Analysis Tool , 2008, 2008 Third International Conference on Availability, Reliability and Security.

[121]  Mehmet Sahinoglu,et al.  An Input–Output Measurable Design for the Security Meter Model to Quantify and Manage Software Security Risk , 2008, IEEE Transactions on Instrumentation and Measurement.

[122]  Richard P. Lippmann,et al.  An Annotated Review of Past Papers on Attack Graphs , 2005 .

[123]  Marcella Lazar,et al.  A security metric for the evaluation of collaborative intrusion detection systems in wireless sensor networks , 2017, 2017 IEEE International Conference on Communications (ICC).

[124]  Klara Nahrstedt,et al.  Quality of protection for mobile multimedia applications , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[125]  Somesh Jha,et al.  Automated generation and analysis of attack graphs , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[126]  Bharat K. Bhargava,et al.  Maximizing network security given a limited budget , 2009, TAPIA '09.

[127]  A. Gani,et al.  Measuring security for cloud service provider: A Third Party approach , 2014, 2013 International Conference on Electrical Information and Communication Technology (EICT).

[128]  Mohammad Abdollahi Azgomi,et al.  A stochastic model of attack process for the evaluation of security metrics , 2013, Comput. Networks.

[129]  Ehab Al-Shaer,et al.  Vulnerability analysis For evaluating quality of protection of security policies , 2006, QoP '06.

[130]  Sushil Jajodia,et al.  k-Zero Day Safety: Measuring the Security Risk of Networks against Unknown Attacks , 2010, ESORICS.

[131]  Ehab Al-Shaer,et al.  A Novel Quantitative Approach For Measuring Network Security , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[132]  Robert K. Cunningham,et al.  Why Measuring Security Is Hard , 2010, IEEE Security & Privacy.

[133]  Sushil Jajodia,et al.  An Attack Graph-Based Probabilistic Security Metric , 2008, DBSec.

[134]  Yu Liu,et al.  Network vulnerability assessment using Bayesian networks , 2005, SPIE Defense + Commercial Sensing.

[135]  Cynthia A. Phillips,et al.  A graph-based system for network-vulnerability analysis , 1998, NSPW '98.

[136]  Mark Merkow,et al.  Computer Security Assurance Using the Common Criteria , 2004 .

[137]  Nora Cuppens-Boulahia,et al.  Towards dynamic risk management: Success likelihood of ongoing attacks , 2012, Bell Labs Technical Journal.

[138]  Ram Dantu,et al.  Risk management using behavior based attack graphs , 2004, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004..

[139]  Stefan Lucks,et al.  Cryptographic Attack Metrics , 2005, Dependability Metrics.