Correlation analysis of intrusion alerts

Security systems such as intrusion detection systems (IDSs) are widely deployed into networks to better protect digital assets. However, there are several problems related to current IDSs. (1) IDSs may flag a large number of alerts everyday, thus overwhelming the security officers. (2) Among the alerts flagged by IDSs, false alerts (i.e., false positives) are mixed with true ones, and usually it is difficult to differentiate between them. (3) Existing IDSs may not detect all attacks launched by adversaries. These problems make it very challenging for human users or intrusion response systems to understand the alerts and take appropriate actions. Thus, it is necessary to perform alert correlation. My dissertation focuses on correlation analysis of intrusion alerts. In particular, I have worked on the following issues. The first issue is the efficiency of alert correlation. This work is extended from our previous correlation method [83]. The initial implementation of [83] is a Database Management System based toolkit. To improve its performance, we propose to adapt main memory index structures and database query optimization techniques to facilitate timely correlation of intensive alerts. We present three techniques named hyper-alert container, two-level index, and sort correlation, and study the performance of these techniques. The second issue is to learn attack strategies. We notice that understanding the strategies of attacks is crucial for security applications such as network forensics and intrusion response. We propose techniques to automatically learn attack strategies from intrusion alerts, where attack strategies are modeled as directed graphs with nodes representing attacks and edges representing constraints between corresponding nodes. We further present techniques to measure the similarity between attack strategies using the techniques in error tolerant graph/subgraph isomorphism. The third issue is how to hypothesize and reason about attacks missed by IDSs. We notice that current alert correlation methods depend heavily on the underlying IDSs for providing alerts, and cannot deal with attacks missed by IDSs. We present techniques to hypothesize attacks possibly missed by the IDSs, to infer attribute values for hypothesized attacks, to validate and prune hypothesized attacks through examining raw audit data, and to consolidate hypothesized attacks to get concise attack scenarios. The fourth issue is to correlate alerts from different security systems. We notice that complementary security systems such as IDSs and firewalls are widely deployed in networks. We propose a correlation approach based on triggering events and common resources. Our approach first performs alert clustering such that the alerts in each cluster share "similar" triggering events. We further propose techniques to build attack scenarios through identifying "common" resources between different attacks. The fifth issue is privacy-privacy alert correlation. We notice that there are privacy concerns when intrusion alerts are shared and correlated among different organizations. We propose one generalization based scheme and three perturbation based schemes to anonymize alerts to protect data privacy. To evaluate privacy protection, we use entropy to guide alert anonymization. In addition, to learn the utility of anonymized alerts, we further perform correlation analysis for anonymized data sets. We focus on estimating similarity values between anonymized attributes and building attack scenarios from anonymized data sets. Finally, the conclusion of my dissertation is provided and future work is pointed out.