A Novel Metric for Measuring Operational Effectiveness of a Cybersecurity Operations Center

Cybersecurity threats are on the rise with evermore digitization of the information that many day-to-day systems depend upon. The demand for cybersecurity analysts outpaces supply, which calls for optimal management of the analyst resource. In this chapter, a new notion of cybersecurity risk is defined, which arises when alerts from intrusion detection systems remain unanalyzed at the end of a work-shift. The above risk poses a security threat to the organization, which in turn impacts the operational effectiveness of the cybersecurity operations center (CSOC). The chapter considers four primary analyst resource parameters that influence risk. For a given risk threshold, the parameters include (1) number of analysts in a work-shift, and in turn within the organization, (2) expertise mix of analysts in a work-shift to investigate a wide range of alerts, (3) optimal sensor to analyst allocation, and (4) optimal scheduling of analysts that guarantees both number and expertise mix of analysts in every work-shift. The chapter presents a thorough treatment of risk and the role it plays in analyst resource management within a CSOC under varying alert generation rates from sensors. A simulation framework to measure risk under various model parameter settings is developed, which can also be used in conjunction with an optimization model to empirically validate the optimal settings of the above model parameters. The empirical results, sensitivity study, and validation study confirms the viability of the framework for determining the optimal management of the analyst resource that minimizes risk under the uncertainty of alert generation and model constraints.

[1]  John McHugh,et al.  Turning Contradictions into Innovations or: How We Learned to Stop Whining and Improve Security Operations , 2016, SOUPS.

[2]  Kasia Muldner,et al.  Preparation, detection, and analysis: the diagnostic work of IT security incident response , 2010, Inf. Manag. Comput. Secur..

[3]  Sushil Jajodia,et al.  Dynamic Scheduling of Cybersecurity Analysts for Minimizing Risk Using Reinforcement Learning , 2016, ACM Trans. Intell. Syst. Technol..

[4]  David Lesaint,et al.  Field workforce scheduling , 2003 .

[5]  Anita D. D'Amico,et al.  The Real Work of Computer Network Defense Analysts , 2007, VizSEC.

[6]  Pratyusa K. Manadhata,et al.  The Operational Role of Security Information and Event Management Systems , 2014, IEEE Security & Privacy.

[7]  Yves Nobert,et al.  Freight Handling Personnel Scheduling at Air Cargo Terminals , 1998, Transp. Sci..

[8]  Reeshad S. Dalal,et al.  Psychosocial Dynamics of Cyber Security , 2016 .

[9]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[10]  Mehmet Emin Aydin,et al.  Dynamic job-shop scheduling using reinforcement learning agents , 2000, Robotics Auton. Syst..

[11]  Kasia Muldner,et al.  Toward understanding distributed cognition in IT security management: the role of cues and norms , 2011, Cognition, Technology & Work.

[12]  Dorothy E. Denning,et al.  An Intrusion-Detection Model , 1987, IEEE Transactions on Software Engineering.

[13]  Tudor Dumitras,et al.  The Global Cyber-Vulnerability Report , 2015, Terrorism, Security, and Computation.

[14]  Tapas K. Das,et al.  A multi-agent reinforcement learning approach to obtaining dynamic control policies for stochastic lot scheduling problem , 2005, Simul. Model. Pract. Theory.

[15]  Christopher Krügel,et al.  Comprehensive approach to intrusion detection alert correlation , 2004, IEEE Transactions on Dependable and Secure Computing.

[16]  Sushil Jajodia,et al.  Applications of Data Mining in Computer Security , 2002, Advances in Information Security.

[17]  Fabio Persia,et al.  Discovering the Top-k Unexplained Sequences in Time-Stamped Observation Data , 2014, IEEE Transactions on Knowledge and Data Engineering.

[18]  Sushil Jajodia,et al.  Optimal Scheduling of Cybersecurity Analysts for Minimizing Risk , 2017, ACM Trans. Intell. Syst. Technol..

[19]  Subbarao Kambhampati,et al.  Planning and Scheduling , 1997, The Computer Science and Engineering Handbook.

[20]  Robert F. Erbacher,et al.  Extending Case-Based Reasoning to Network Alert Reporting , 2012, 2012 International Conference on Cyber Security.

[21]  Fuqing Zhao,et al.  A Dynamic Rescheduling Model with Multi-Agent System and Its Solution Method , 2012 .

[22]  Carl M. Harris,et al.  Fundamentals of Queueing Theory: Gross/Fundamentals of Queueing Theory , 2008 .

[23]  Nuno J. Mamede,et al.  Multi-Agent Dynamic Scheduling and Re-Scheduling with Global Temporal Constraints , 2001, ICEIS.

[24]  Ali Ghorbani,et al.  Alert correlation survey: framework and techniques , 2006, PST.

[25]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[26]  Shanchieh Jay Yang,et al.  Temporal and Spatial Analyses for Large-Scale Cyber Attacks , 2013 .