Measurement and spectral analysis of denial of service attacks

Denial of service (DoS) attacks cause significant financial damage every year, making it essential to devise techniques to detect and respond to attacks quickly. Although many protection systems have been proposed by the research and commercial communities the problem remains largely unsolved. We believe insight into attack stream dynamics---the interaction of malicious packets with the network will aid in the development of more robust next generation attack detection and response systems. This thesis combines two traditionally separate fields, computer systems and statistical signal processing, to understand attack stream dynamics and develop novel analysis techniques. In order to have a representation dataset for analysis, we deploy a trace collection system to capture real-world DoS attacks. We then propose unique attack classification and detection methodologies using statistical signal processing techniques to analyze attack stream behavior. First, we develop an automated methodology for characterizing DoS attacks into single and multi-source attacks. Our methodology proposes new techniques of ramp-up and spectral analysis building on the existing approach of packet header analysis to robustly characterize attacks. This framework can be used as part of an automated DoS detection and response system to aid network administrators in selecting an appropriate response. Second, using a combination of statistical signal processing and pattern recognition techniques, we develop an attack fingerprinting system that provides the ability to identify repeated attacks. Fingerprints not only aid in attribution for criminal and civil prosecution of attacker, but also help justify response measures and quantify DoS activity. Finally, we propose a wavelet-based attack detection system that allows detection of low bandwidth attacks in aggregate network traffic. This technique is more sensitive and proactive than current approaches and moves filtering from the victim to the attacker sources where attacks can be terminated quickly. In this dissertation we show that the attack traffic inherently has periodicities encoded in the packet stream that can be analyzed in order to characterize attacks. Although the analysis techniques are developed primarily to analyze attack stream behavior, they can be directly applied to analyze periodic behavior in a range of other network analysis problems.