Data Fusion of Security Logs to Measure Critical Security Controls to Increase Situation Awareness

In Jan. 2018, a NIST draft to the Cybersecurity Framework called for the development of cybersecurity metrics, saying such work would be a “major advancement and contribution to the cybersecurity community (National Institute of Standards and Technology, 2017b).” Unfortunately, organizations and researchers continue to make little progress at measuring security. Along with this, research around measuring security fails to present detailed guides on how to implement security metrics collection and reporting in an organization. This research seeks to explore how measuring the CIS (formally SANS) Critical Security Controls, through data fusion of security logs, has the potential to increase situation awareness to strategic decision makers, and systems administrators. Metrics are built for each of the sub controls for Critical Security Control 8: Malware Defenses. Along with the development of these metrics, a proof of concept is implemented in a computer network designed to mimic a small business that is using Symantec Endpoint Protection and Splunk. A Splunk dashboard is created to monitor, in real time, the status of Critical Security Control 8.1 and 8.2. A discussion on the actionable information and value provided by these dashboards occurs. This work contributes to the industry’s need for cybersecurity metrics through the development of six metrics. Along with this, a detailed implementation guide is provided for security practitioners looking to implement metrics for Critical Security Controls 8.1 and 8.2 in

[1]  F. Baiardi,et al.  Metrics for Cyber Robustness , 2017 .

[2]  Eric Ouellet,et al.  Magic Quadrant for Endpoint Protection Platforms , 2013 .

[3]  Ronald S. Ross,et al.  Protecting Controlled Unclassified Information in Nonfederal Information Systems and Organizations [including updates as of 02-20-2018] | NIST , 2017 .

[4]  Lori Homsher Gathering Security Metrics and Reaping the Rewards , 2009 .

[5]  Karen A. Scarfone,et al.  Cyber Security Metrics and Measures , 2008 .

[6]  Ulrik Franke,et al.  Cyber situational awareness - A systematic review of the literature , 2014, Comput. Secur..

[7]  Robin Sommer,et al.  Bro: An Open Source Network Intrusion Detection System , 2003, DFN-Arbeitstagung über Kommunikationsnetze.

[8]  Monica Mehrotra,et al.  Security Issue - A Metrics Perspective , 2010 .

[9]  M R Endsley,et al.  Sources of situation awareness errors in aviation. , 1996, Aviation, space, and environmental medicine.

[10]  Joint Task Force Transformation Initiative,et al.  Security and Privacy Controls for Federal Information Systems and Organizations , 2013 .

[11]  Daniel J. Garland,et al.  Situation Awareness Analysis and Measurement , 2009 .

[12]  Daniel R. Tesone,et al.  Achieving Cyber Defense Situational Awareness: A Cognitive Task Analysis of Information Assurance Analysts , 2005 .

[13]  Tim Bass,et al.  Intrusion detection systems and multisensor data fusion , 2000, CACM.

[14]  Dirk Draheim On the Design of IT Key Performance Indicators , 2011, 2011 22nd International Workshop on Database and Expert Systems Applications.

[15]  Michael D. McNeese,et al.  Information data fusion and computer network defense , 2012 .

[16]  Shirley C. Payne,et al.  A Guide to Security Metrics , 2007 .

[17]  Nicklaus A. Giacobe,et al.  Application of the JDL data fusion process model for cyber security , 2010, Defense + Commercial Sensing.

[18]  Sushil Jajodia,et al.  k-Zero Day Safety: A Network Security Metric for Measuring the Risk of Unknown Vulnerabilities , 2014, IEEE Transactions on Dependable and Secure Computing.

[19]  Kevin M. Stine,et al.  Performance Measurement Guide for Information Security , 2008 .

[20]  David L. Hall,et al.  New perspectives on level-5 information fusion: The impact of advances in information technology and user behavior , 2015, 2015 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI).

[21]  John G. Voeller Wiley handbook of science and technology for homeland security , 2008 .

[22]  Luigi Coppolino,et al.  Enabling Convergence of Physical and Logical Security Through Intelligent Event Correlation , 2015, IDC.

[23]  Eric Michael Hutchins,et al.  Intelligence-Driven Computer Network Defense Informed by Analysis of Adversary Campaigns and Intrusion Kill Chains , 2010 .

[24]  Lalu Banoth,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2017 .

[25]  William A. Wulf,et al.  TOWARDS A FRAMEWORK FOR SECURITY MEASUREMENT , 1997 .

[26]  Reijo Savola,et al.  Towards a Security Metrics Taxonomy for the Information and Communication Technology Industry , 2007, International Conference on Software Engineering Advances (ICSEA 2007).

[27]  Wayne A. Jansen,et al.  Directions in Security Metrics Research , 2009 .

[28]  Andy Ju An Wang Information security models and metrics , 2005, ACM-SE 43.

[29]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[30]  Nicklaus A. Giacobe A Picture is Worth a Thousand Alerts , 2013 .

[31]  Fabio Martinelli,et al.  Formal approach to security metrics.: what does "more secure" mean for you? , 2010, ECSA '10.

[32]  Hanno Langweg,et al.  Framework for malware resistance metrics , 2006, QoP '06.

[33]  Suzanne P. Hassell,et al.  Measurement, identification and calculation of cyber defense metrics , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[34]  Marianne Swanson,et al.  Security metrics guide for information technology systems , 2003 .

[35]  Nicklaus A. Giacobe Data fusion in cyber security: first order entity extraction from common cyber data , 2012, Defense + Commercial Sensing.

[36]  Pascal Vasseur,et al.  Introduction to Multisensor Data Fusion , 2005, The Industrial Information Technology Handbook.

[37]  Hao Wang,et al.  Security metrics for software systems , 2009, ACM-SE 47.

[38]  George Cybenko Quantifying and measuring cyber resiliency , 2016, SPIE Defense + Security.

[39]  Jouko Vankka,et al.  Situational awareness and information collection from critical infrastructure , 2014, 2014 6th International Conference On Cyber Conflict (CyCon 2014).

[40]  Risto Vaarandi,et al.  Using Security Logs for Collecting and Reporting Technical Security Metrics , 2014, 2014 IEEE Military Communications Conference.

[41]  Gregory A. Witte,et al.  Framework for Improving Critical Infrastructure Cybersecurity | NIST , 2014 .

[42]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[43]  Neeraj Suri,et al.  A security metrics framework for the Cloud , 2011, Proceedings of the International Conference on Security and Cryptography.

[44]  Alan N. Steinberg,et al.  Revisions to the JDL data fusion model , 1999, Defense, Security, and Sensing.

[45]  M. Tyworth,et al.  The distributed nature of cyber situation awareness , 2012, 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support.

[46]  Ralph L. Keeney,et al.  Selecting Attributes to Measure the Achievement of Objectives , 2005, Oper. Res..

[47]  Erik Blasch,et al.  JDL level 5 fusion model: user refinement issues and applications in group tracking , 2002, SPIE Defense + Commercial Sensing.

[48]  Maya Ingle,et al.  A Review of Security Metrics in Software Development Process , 2011 .

[49]  Marco Casassa Mont,et al.  Using security metrics coupled with predictive modeling and simulation to assess security processes , 2009, 2009 3rd International Symposium on Empirical Software Engineering and Measurement.

[50]  Mario Piattini,et al.  A comparison of software design security metrics , 2010, ECSA '10.

[51]  M. McNeese,et al.  idsNETS: An experimental platform to study situation awareness for intrusion detection analysts , 2012, 2012 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support.

[52]  Murray Turoff,et al.  The Delphi Method: Techniques and Applications , 1976 .

[53]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .