Integration of Self-Organizing Map (SOM) and Kernel Density Estimation (KDE) for network intrusion detection

This paper proposes an approach to integrate the self-organizing map (SOM) and kernel density estimation (KDE) techniques for the anomaly-based network intrusion detection (ABNID) system to monitor the network traffic and capture potential abnormal behaviors. With the continuous development of network technology, information security has become a major concern for the cyber system research. In the modern net-centric and tactical warfare networks, the situation is more critical to provide real-time protection for the availability, confidentiality, and integrity of the networked information. To this end, in this work we propose to explore the learning capabilities of SOM, and integrate it with KDE for the network intrusion detection. KDE is used to estimate the distributions of the observed random variables that describe the network system and determine whether the network traffic is normal or abnormal. Meanwhile, the learning and clustering capabilities of SOM are employed to obtain well-defined data clusters to reduce the computational cost of the KDE. The principle of learning in SOM is to self-organize the network of neurons to seek similar properties for certain input patterns. Therefore, SOM can form an approximation of the distribution of input space in a compact fashion, reduce the number of terms in a kernel density estimator, and thus improve the efficiency for the intrusion detection. We test the proposed algorithm over the real-world data sets obtained from the Integrated Network Based Ohio University's Network Detective Service (INBOUNDS) system to show the effectiveness and efficiency of this method.

[1]  M. C. Jones,et al.  On a class of kernel density estimate bandwidth selectors , 1991 .

[2]  M. C. Jones,et al.  A Brief Survey of Bandwidth Selection for Density Estimation , 1996 .

[3]  D. W. Scott,et al.  Biased and Unbiased Cross-Validation in Density Estimation , 1987 .

[4]  Hujun Yin,et al.  Self-organizing mixture networks for probability density estimation , 2001, IEEE Trans. Neural Networks.

[5]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[6]  Stephan R. Sain,et al.  Multi-dimensional Density Estimation , 2004 .

[7]  Hiroyuki Kitagawa,et al.  Outlier Detection for Transaction Databases Using Association Rules , 2008, 2008 The Ninth International Conference on Web-Age Information Management.

[8]  S. E. Smaha.  Haystack: an intrusion detection system , 1988, [Proceedings 1988] Fourth Aerospace Computer Security Applications.

[9]  Alfonso Valdes,et al.  Next-generation Intrusion Detection Expert System (NIDES)A Summary , 1997 .

[10]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[11]  Monica C. Jackson,et al.  Introduction to the Practice of Statistics , 2001 .

[12]  Peter G. Neumann,et al.  EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances , 1997, CCS 2002.

[13]  Eugene H. Spafford,et al.  An Application of Pattern Matching in Intrusion Detection , 1994 .

[14]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.

[15]  M. C. Jones,et al.  On optimal data-based bandwidth selection in kernel density estimation , 1991 .

[16]  Ramani Duraiswami,et al.  Fast optimal bandwidth selection for kernel density estimation , 2006, SDM.

[17]  Hongxing He,et al.  Outlier Detection Using Replicator Neural Networks , 2002, DaWaK.

[18]  James Stephen Marron,et al.  Comparison of data-driven bandwith selectors , 1988 .

[19]  Udo W. Pooch,et al.  Cooperating security managers: Distributed intrusion detection systems , 1996, Comput. Secur..

[20]  Philip S. Yu,et al.  Outlier detection for high dimensional data , 2001, SIGMOD '01.

[21]  Marina Blanton,et al.  Real-Time Network-Based Anomaly Intrusion Detection , 2001, Scalable Comput. Pract. Exp..

[22]  Biswanath Mukherjee,et al.  A network security monitor , 1990, Proceedings. 1990 IEEE Computer Society Symposium on Research in Security and Privacy.

[23]  A. Bowman.  An alternative method of cross-validation for the smoothing of density estimates , 1984 .