Increasing reliability in network traffic anomaly detection

Network traffic anomalies stand for a large fraction of the Internet traffic and compromise the performance of the network resources. Detecting and diagnosing these threats is a laborious and time consuming task that network operators face daily. During the last decade researchers have concentrated their efforts on this problem and proposed several tools to automate this task. Thereby, recent advances in anomaly detection have permitted to detect new or unknown anomalies by taking advantage of statistical analysis of the traffic. In spite of the advantages of these detection methods, researchers have reported several common drawbacks discrediting their use in practice. Indeed, the challenge of understanding the relation between the theory underlying these methods and the actual Internet traffic raises several issues. For example, the difficulty of selecting the optimal parameter set for these methods mitigates their performance and prevent network operators from using them. Moreover, due to the lack of ground truth data, approximate evaluations of these detection methods prevent to provide accurate feedback on them and increase their reliability. We address these issues, first, by proposing a pattern-recognition-based detection method that overcomes the common drawbacks of anomaly detectors based on statistical analysis, second, by providing both a benchmark tool that compares the results from diverse detectors and ground truth data obtained by combining several anomaly detectors. The proposed pattern-recognition-based detector takes advantage of image processing techniques to provide intuitive outputs and parameter set. An adaptive mechanism automatically tuning its parameter set according to traffic fluctuations is also proposed. The resulting adaptive anomaly detector is easily usable in practice, performs a high detection rate, and provides intuitive description of the anomalies allowing to identify their root causes. A benchmark methodology is also developed in order to compare several detectors based on different theoretical background. This methodology allows researchers to accurately identify the differences between the results of diverse detectors. We employ this methodology along with an unsupervised combination strategy to combine the output of four anomaly detectors. Thereby, the combination strategy increases the overall reliability of the combined detectors and it detects two times more anomalies than the best detector. We provide the results of this combination of detectors in the form of ground truth data containing various anomalies during 10 years of traffic.