Anomaly Detection by Clustering in the Network

Intrusions impose serious security threat to network environment, so it is necessary to detect and cope with them. Many intrusion detection methods focus on signature detection, where models are built to recognize known attacks. However, signature detection, limited by its nature, cannot detect novel attacks. New intrusion types, of which detection systems may not even be aware, are difficult to detect. Anomaly detection focuses on modeling the normal behavior and identifying significant deviations, which could be novel attacks. In this paper we present a clustering algorithm to identify outliers. It performs clustering on feature vectors collected from the network and can automatically detect new types of intrusions without need of manual classification of training data. Experimental results show that our system achieves a satisfactory intrusions detection rate while keeping the false positive rate reasonably low.

[1]  H. Javitz,et al.  Detecting Unusual Program Behavior Using the Statistical Component of the Next-generation Intrusion Detection Expert System ( NIDES ) 1 , 1997 .

[2]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[3]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD 2000.

[4]  Michael Schatz,et al.  Learning Program Behavior Profiles for Intrusion Detection , 1999, Workshop on Intrusion Detection and Network Monitoring.

[5]  Carla E. Brodley,et al.  Temporal sequence learning and data reduction for anomaly detection , 1998, CCS '98.

[6]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[7]  Peter G. Neumann,et al.  Experience with EMERALD to Date , 1999, Workshop on Intrusion Detection and Network Monitoring.

[8]  Mohammed J. Zaki,et al.  ADMIT: anomaly-based data mining for intrusions , 2002, KDD.

[9]  Sridhar Ramaswamy,et al.  Efficient algorithms for mining outliers from large data sets , 2000, SIGMOD '00.

[10]  R. Sekar,et al.  A fast automaton-based method for detecting anomalous program behaviors , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[11]  Philip S. Yu,et al.  Outlier detection for high dimensional data , 2001, SIGMOD '01.

[12]  Raymond T. Ng,et al.  Algorithms for Mining Distance-Based Outliers in Large Datasets , 1998, VLDB.

[13]  Stephanie Forrest,et al.  Computer immunology , 1997, CACM.

[14]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[15]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[16]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[17]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[18]  Sushil Jajodia,et al.  Detecting Novel Network Intrusions Using Bayes Estimators , 2001, SDM.

[19]  ShimKyuseok,et al.  Efficient algorithms for mining outliers from large data sets , 2000 .