Statistical profiling and visualization for detection of malicious insider attacks on computer networks

The massive volume of intrusion detection system (IDS) alarms generated on large networks, and the resulting need for labor-intensive security analysis of the text-based IDS alarm logs, has recently brought into question the cost-effectiveness of IDSs. In particular, when host-based IDSs are used to monitor an organization's internal networks, the majority of the resulting alarms represent legitimate, automated system administration. Because of the absence of ground truth about known attacks, we propose an unsupervised, anomaly-based method for automatically distinguishing alarms that are potentially generated by malicious insider attacks, from the repetitive and temporally structured legitimate system-administration alarms. The majority of previous work in this area has used heuristic and statistical filtering techniques to discard a relatively large proportion of alarms in the final presentation to the security analyst, which is a potentially dangerous practice. Instead, we demonstrate the use of a typicality measure to visualize the apparent risk associated with alarms, while retaining information about the temporal context of the entire alarm stream for the analyst to view. The relevance of the statistical method is examined by comparing the results to a set of analyst-curated alarms from an operational environment.