A framework for Operational Security Metrics Development for industrial control environment

ABSTRACT Security metrics are very crucial towards providing insights when measuring security states and susceptibilities in industrial operational environments. Obtaining practical security metrics depend on effective security metrics development approaches. To be effective, a security metrics development framework should be scope-definitive, objective-oriented, reliable, simple, adaptable, and repeatable (SORSAR). A framework for Operational Security Metrics Development (OSMD) for industry control environments is presented, which combines concepts and characteristics from existing approaches. It also adds the new characteristic of adaptability. The OSMD framework is broken down into three phases of: target definition, objective definition, and metrics synthesis. A case study scenario is used to demonstrate an instance of how to implement and apply the proposed framework to demonstrate its usability and workability. Expert elicitation has also be used to consolidate the validity of the proposed framework. Both validation approaches have helped to show that the proposed framework can help create effective and efficient ICS-centric security metrics taxonomy that can be used to evaluate capabilities or vulnerabilities. The understanding from this can help enhance security assurance within industrial operational environments.

[1]  Joint Task Force Transformation Initiative,et al.  Security and Privacy Controls for Federal Information Systems and Organizations , 2013 .

[2]  Dirk Draheim.  On the Design of IT Key Performance Indicators , 2011, 2011 22nd International Workshop on Database and Expert Systems Applications.

[3]  Reijo Savola.  A Security Metrics Development Method for Software Intensive Systems , 2009 .

[4]  Juhee Kwon,et al.  Proactive vs. Reactive Security Investments in the Healthcare Sector Proactive vs. Reactive Security Investments in the Healthcare Sector , 2011 .

[5]  Ping Wang,et al.  Threat Analysis of Cyber Attacks with Attack Tree+ , 2014, J. Inf. Hiding Multim. Signal Process..

[6]  Austen Rainer,et al.  Using an expert panel to validate a requirements process improvement model , 2005, J. Syst. Softw..

[7]  Zachary A. Collier,et al.  Security Metrics in Industrial Control Systems , 2015, ArXiv.

[8]  Michael R. Grimaila,et al.  Evaluation of security solutions in the SCADA environment , 2014, DATB.

[9]  Mark E. Borsuk,et al.  Concepts of decision support for river rehabilitation , 2007, Environ. Model. Softw..

[10]  Jack P. C. Kleijnen,et al.  EUROPEAN JOURNAL OF OPERATIONAL , 1992 .

[11]  George Cybenko.  TIM Lecture Series Cybersecurity Metrics and Simulation , 2014 .

[12]  Shari Lawrence Pfleeger,et al.  An empirical study of maintenance and development estimation accuracy , 2002, J. Syst. Softw..

[13]  Habtamu Abie,et al.  Development of security metrics for a distributed messaging system , 2009, 2009 International Conference on Application of Information and Communication Technologies.

[14]  Reijo Savola,et al.  Development of Measurable Security for a Distributed Messaging System , 2010 .

[15]  T. Chandrakumar,et al.  Security Metrics for a Business Information System , 2013 .

[16]  Stefano Bistarelli,et al.  Evaluation of complex security scenarios using defense trees and economic indexes , 2012, J. Exp. Theor. Artif. Intell..

[17]  Shahrin Sahib,et al.  TECHNICAL SECURITY METRICS MODEL IN COMPLIANCE WITH ISO/IEC 27001 STANDARD , 2012 .

[18]  Miles McQueen,et al.  Measurable Control System Security through Ideal Driven Technical Metrics , 2008 .

[19]  Jennifer Shu-Jen Lin,et al.  Planning Horizon for Production Inventory Models with Production Rate Dependent on Demand and Inventory Level , 2013, J. Appl. Math..

[20]  Philip O’Neill.  Protecting Critical Infrastructure by Identifying Pathways of Exposure to Risk , 2013 .

[21]  Frank Yeong-Sung Lin,et al.  Effective Proactive and Reactive Defense Strategies against Malicious Attacks in a Virtualized Honeynet , 2013, J. Appl. Math..

[22]  Ali Mili,et al.  Quantifying availability in SCADA environments using the cyber security metric MFC , 2014, CISR '14.

[23]  Suku Nair,et al.  Cyber Security Analytics: A Stochastic Model for Security Quantification Using Absorbing Markov Chains , 2014 .

[24]  Tony Rosqvist,et al.  Software Quality Evaluation Based on Expert Judgement , 2003, Software Quality Journal.

[25]  Ashutosh Tiwari,et al.  Human Capability Evaluation Approach for Cyber Security in Critical Industrial Infrastructure , 2016 .

[26]  Khaled El Emam,et al.  An instrument for measuring the success of the requirements engineering process in information systems development , 2004, Empirical Software Engineering.

[27]  David J. Musliner,et al.  Self-Adaptation Metrics for Active Cybersecurity , 2013, 2013 IEEE 7th International Conference on Self-Adaptation and Self-Organizing Systems Workshops.

[28]  Venkatesh Jaganathan,et al.  Using a Prediction Model to Manage Cyber Security Threats , 2015, TheScientificWorldJournal.

[29]  Sushil Jajodia,et al.  Topological analysis of network attack vulnerability , 2006, PST.

[30]  Tore Dybå,et al.  An Instrument for Measuring the Key Factors of Success in Software Process Improvement , 2000, Empirical Software Engineering.

[31]  Marco Casassa Mont,et al.  Using security metrics coupled with predictive modeling and simulation to assess security processes , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[32]  Erland Jonsson,et al.  A Framework for Security Metrics Based on Operational System Attributes , 2011, 2011 Third International Workshop on Security Measurements and Metrics.

[33]  Karen A. Scarfone,et al.  Guide to Industrial Control Systems (ICS) Security , 2015 .

[34]  Marco Cremonini,et al.  Evaluating Information Security Investments from Attackers Perspective: the Return-On-Attack (ROA) , 2005, WEIS.

[35]  Suku Nair,et al.  Exploitability analysis using predictive cybersecurity framework , 2015, 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF).

[36]  Erland Jonsson,et al.  Towards an integrated conceptual model of security and dependability , 2006, First International Conference on Availability, Reliability and Security (ARES'06).

[37]  Khaled El Emam,et al.  Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability , 2000, IEEE Trans. Software Eng..

[38]  Stefan Biffl,et al.  Addressing misalignment between information security metrics and business-driven security objectives , 2010, MetriSec '10.

[39]  Reijo Savola,et al.  Towards a Security Metrics Taxonomy for the Information and Communication Technology Industry , 2007, International Conference on Software Engineering Advances (ICSEA 2007).

[40]  Igor Linkov,et al.  Measurable resilience for actionable policy. , 2013, Environmental science & technology.

[41]  Alexander Kott,et al.  Cyber-security of SCADA and Other Industrial Control Systems , 2016, Advances in Information Security.

[42]  Barbara A. Kitchenham,et al.  A framework for evaluating a software bidding model , 2005, Inf. Softw. Technol..

[43]  Y. Lei,et al.  Vertical Track Irregularity Influence on the Wheel High-Frequency Vibration in Wheel-Rail System , 2016 .

[44]  William H. Sanders,et al.  Model-based Security Metrics Using ADversary VIew Security Evaluation (ADVISE) , 2011, 2011 Eighth International Conference on Quantitative Evaluation of SysTems.

[45]  R. K. Shyamasundar,et al.  Security and protection of SCADA: a bigdata algorithmic approach , 2013, SIN.

[46]  Steven M. Bellovin.  On the Brittleness of Software and the Infeasibility of Security Metrics , 2006, IEEE Security & Privacy Magazine.

[47]  S. Kowalski,et al.  SECURITY METRICS AND EVALUATION OF INFORMATION SYSTEMS SECURITY , 2004 .

[48]  Igor Linkov,et al.  Resilience metrics for cyber systems , 2013, Environment Systems and Decisions.

[49]  Duohe Ma,et al.  Moving Target Network Defense Effectiveness Evaluation Based on Change-Point Detection , 2016 .

[50]  Igor V. Kotenko,et al.  A Cyber Attack Modeling and Impact Assessment framework , 2013, 2013 5th International Conference on Cyber Conflict (CYCON 2013).

[51]  Stefano Bistarelli,et al.  Defense trees for economic evaluation of security investments , 2006, First International Conference on Availability, Reliability and Security (ARES'06).

[52]  S. Shahrin,et al.  A propose technical security metrics model for SCADA systems , 2012, Proceedings Title: 2012 International Conference on Cyber Security, Cyber Warfare and Digital Forensic (CyberSec).

[53]  Cumberland Emergency,et al.  Framework for Improving Critical Infrastructure Cybersecurity News From Down Under , 2014 .

[54]  Stewart Robinson,et al.  Simulation model verification and validation: increasing the users' confidence , 1997, WSC '97.

[55]  Søren Lauesen,et al.  Preventing Requirement Defects: An Experiment in Process Improvement , 2001, Requirements Engineering.

[56]  Juhee Kwon,et al.  Proactive Versus Reactive Security Investments in the Healthcare Sector , 2014, MIS Q..

[57]  Shirley C. Payne,et al.  A Guide to Security Metrics , 2007 .

[58]  Marjan Keramati,et al.  Novel security metrics for ranking vulnerabilities in computer networks , 2014, 7'th International Symposium on Telecommunications (IST'2014).

[59]  Habtamu Abie,et al.  Metrics-driven security objective decomposition for an e-health application with adaptive security management , 2013, ASPI '13.