A spatiotemporal event correlation approach to computer security

Correlation is a recognized technique in security to improve the effectiveness of threat identification and analysis process. Existing correlation approaches mostly focus on correlating temporally located events, or combining alerts from multiple intrusion detection systems. Such approaches either generate high false alarm rates due to single host activity changes, or fail to detect stealthy attacks that evade detection from local monitors. This thesis explores a new spatiotemporal event correlation approach to capture the abnormal patterns of a wide class of attacks, whose activities, when observed individually, may not seem suspicious or distinguishable from normal activity changes. This approach correlates events across both space and time, identifying aggregated abnormal event patterns to the host state updates. By exploring both the temporal and spatial locality of host state changes, our approach identifies malicious events that are hard to detect in isolation, without foreknowledge of normal changes or system-specific knowledge. To demonstrate the effectiveness of spatiotemporal event correlation, we instantiate the approach in two example security applications: anomaly detection and network forensics. For anomaly detection, we present a "pointillist" method to detect similar, coincident changes to the patterns of file updates that are shared across multiple hosts. The correlation is performed by clustering points, each representing an individual host state transition, in a multi-dimensional feature space. We implement this approach in a prototype system called Seurat and demonstrate its effectiveness using a combination of real workstation traces, simulated attacks, and manually launched real worms. For network forensics, we present a general forensics framework called Dragnet, and propose a "random moonwalk" technique that can determine both the host responsible for originating a worm attack and the set of attack flows that make up the initial stages of the attack tree via which the worm infected successive generations of victims. Our technique exploits the "wide tree" shape of a worm propagation by performing random walks backward in time along paths of flows. Using analysis, simulation, and experiments with real world traces, we show how the technique works against both today's fast propagating worms and a wide class of stealthy worms that attempt to hide their attack flows among background traffic. While the high level idea is the same, the two applications use different types of event data, different data representations, and different correlation algorithms, suggesting that spatiotemporal event correlation will be a general solution to reliably and effectively capture the global abnormal patterns for a wide variety of security applications.