Building a Machine Learning Model for the SOC, by the Input from the SOC, and Analyzing it for the SOC

This work demonstrates an ongoing effort to employ and explain machine learning model predictions for classifying alerts in Security Operations Centers (SOC). Our ultimate goal is to reduce analyst workload by automating the process of decision making for investigating alerts using the machine learning model in cases where we can completely trust the model. This way, SOC analysts will be able to focus their time and effort to investigate more complex cases of security alerts. To achieve this goal, we developed a system that shows the prediction for an alert and the prediction explanation to security analysts during their daily workflow of investigating individual security alerts. Another part of our system presents the aggregated model analytics to the managers and stakeholders to help them understand the model and decide, on when to trust the model and let the model make the final decision. Using our prediction explanation visualization, security analysts will be able to classify oncoming alerts more efficiently and gain insight into how a machine learning model generates predictions. Our model performance analysis dashboard helps decision makers analyze the model in signature level granularity and gain more insights about the model.

[1]  Erhan Guven,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2016, IEEE Communications Surveys & Tutorials.

[2]  Lisa Singh,et al.  G-PARE: A visual analytic tool for comparative analysis of uncertain graphs , 2011, 2011 IEEE Conference on Visual Analytics Science and Technology (VAST).

[3]  David H. Tobey,et al.  Enhancing the Cybersecurity Workforce , 2011, IT Professional.

[4]  Yindalon Aphinyanagphongs,et al.  A Workflow for Visual Diagnostics of Binary Classifiers using Instance-Level Explanations , 2017, 2017 IEEE Conference on Visual Analytics Science and Technology (VAST).

[5]  Jarke J. van Wijk,et al.  Understanding the context of network traffic alerts , 2016, 2016 IEEE Symposium on Visualization for Cyber Security (VizSec).

[6]  Arun Lakhotia,et al.  Malware and Machine Learning , 2015, Intelligent Methods for Cyber Warfare.

[7]  Ajmal Mian,et al.  Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey , 2018, IEEE Access.

[8]  Avi Goldfarb,et al.  Prediction Machines: The Simple Economics of Artificial Intelligence , 2018 .

[9]  Andrew M'manga,et al.  Folk Risk Analysis: Factors Influencing Security Analysts' Interpretation of Risk , 2017, SOUPS.

[10]  Giuseppe Santucci,et al.  The goods, the bads and the uglies: Supporting decisions in malware detection through visual analytics , 2017, 2017 IEEE Symposium on Visualization for Cyber Security (VizSec).

[11]  Min Chen,et al.  An Analysis of Machine- and Human-Analytics in Classification , 2017, IEEE Transactions on Visualization and Computer Graphics.