Histogram cloning and CuSum: An experimental comparison between different approaches to Anomaly Detection

Due to the proliferation of new threats from spammers, attackers, and criminal enterprises, Anomaly-based Intrusion Detection Systems have emerged as a key element in network security and different statistical approaches have been considered in the literature. To cope with scalability issues, random aggregation through the use of sketches seems to be a powerful prefiltering stage that can be applied to backbone data traffic. In this paper we compare two different statistical methods to detect the presence of anomalies from such aggregated data. In more detail, histogram cloning (with different distance measurements) and CuSum algorithm (at the bucket level) are tested over A well-known publicly available data set. The performance analysis, presented in this paper, demonstrates the effectiveness of the CuSum when a proper definition of the algorithm, which takes into account the standard deviation of the underlying variables, is chosen.

[1]  Mark Crovella,et al.  Mining anomalies using traffic feature distributions , 2005, SIGCOMM '05.

[2]  Graham Cormode,et al.  An improved data stream summary: the count-min sketch and its applications , 2004, J. Algorithms.

[3]  Kensuke Fukuda,et al.  Extracting hidden anomalies using sketch and non Gaussian multiresolution statistical detection procedures , 2007, LSAD '07.

[4]  Athanasios V. Vasilakos,et al.  DTRAB: Combating Against Attacks on Encrypted Protocols Through Traffic-Feature Analysis , 2010, IEEE/ACM Transactions on Networking.

[5]  Kang G. Shin,et al.  SYN-dog: sniffing SYN flooding sources , 2002, Proceedings 22nd International Conference on Distributed Computing Systems.

[6]  Mikkel Thorup,et al.  Tabulation based 4-universal hashing with applications to second moment estimation , 2004, SODA '04.

[7]  Kensuke Fukuda,et al.  Combining sketch and wavelet models for anomaly detection , 2010, Proceedings of the 2010 IEEE 6th International Conference on Intelligent Computer Communication and Processing.

[8]  Yan Chen,et al.  Reversible sketches for efficient and accurate change detection over network data streams , 2004, IMC '04.

[9]  Vasilios A. Siris,et al.  Application of anomaly detection algorithms for detecting SYN flooding attacks , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[10]  Xenofontas A. Dimitropoulos,et al.  Histogram-based traffic anomaly detection , 2009, IEEE Transactions on Network and Service Management.

[11]  Guanhua Yan,et al.  Blue-Watchdog: Detecting Bluetooth worm propagation in public areas , 2009, 2009 IEEE/IFIP International Conference on Dependable Systems & Networks.

[12]  Ilkka Nomos.  On the Use of Fractional Brownian Motion in the Theory of Connectionless Networks , 1995 .

[13]  Christian Callegari,et al.  Forecasting the Distribution of Network Traffic for Anomaly Detection , 2011, 2011IEEE 10th International Conference on Trust, Security and Privacy in Computing and Communications.

[14]  Osman Salem,et al.  A scalable, efficient and informative approach for anomaly‐based intrusion detection systems: theory and practice , 2010, Int. J. Netw. Manag..

[15]  Jian Kang,et al.  Application Entropy Theory to Detect New Peer-to-Peer Botnet with Multi-chart CUSUM , 2009, 2009 Second International Symposium on Electronic Commerce and Security.

[16]  Marina Thottan,et al.  Anomaly Detection Approaches for Communication Networks , 2010, Algorithms for Next Generation Networks.

[17]  Andrew Clark,et al.  Effective Change Detection in Large Repositories of Unsolicited Traffic , 2009, 2009 Fourth International Conference on Internet Monitoring and Protection.

[18]  Christian Callegari,et al.  A Methodological Overview on Anomaly Detection , 2013, Data Traffic Monitoring and Analysis.

[19]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[20]  Christian Callegari,et al.  On the use of sketches and wavelet analysis for network anomaly detection , 2010, IWCMC.

[21]  Ilkka Norros,et al.  On the Use of Fractional Brownian Motion in the Theory of Connectionless Networks , 1995, IEEE J. Sel. Areas Commun..

[22]  Philippe Flajolet,et al.  Probabilistic Counting Algorithms for Data Base Applications , 1985, J. Comput. Syst. Sci..

[23]  Richard R. Brooks,et al.  Wavelet based Denial-of-Service detection , 2006, Comput. Secur..

[24]  Kavé Salamatian,et al.  Anomaly extraction in backbone networks using association rules , 2009, IMC '09.

[25]  Patrick P. C. Lee,et al.  On the detection of signaling DoS attacks on 3G/WiMax wireless networks , 2009, Comput. Networks.

[26]  Divesh Srivastava,et al.  Holistic UDAFs at streaming speeds , 2004, SIGMOD '04.

[27]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[28]  Rudolf B. Blazek,et al.  Detection of intrusions in information systems by sequential change-point methods , 2005 .

[29]  Christian Callegari,et al.  When randomness improves the anomaly detection performance , 2010, 2010 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies (ISABEL 2010).

[30]  Kensuke Fukuda,et al.  Seven Years and One Day: Sketching the Evolution of Internet Traffic , 2009, IEEE INFOCOM 2009.