Feature Construction Scheme for Efficient Intrusion Detection System

For computationally efficient and effective IDS, it is essential to identify important input features. In this paper, a statistical feature construction scheme is proposed in which factor analysis is orthogonally combined with an optimized k-means clustering technique. As a core component for unsupervised anomaly detection, the proposed feature construction scheme is able to exclude the redundancy of features optimally via the consideration of the similarity of feature responses through a clustering analysis based on the feature space reduced in a factor analysis. The performance of the proposed method was evaluated using different data sets reduced by the ranking of the importance of input features. Experimental results show a significant detection rate through a good subset of features deemed to be critical to the improvement of the performance of classifiers.

[1]  A.H. Sung,et al.  Identifying important features for intrusion detection using support vector machines and neural networks , 2003, 2003 Symposium on Applications and the Internet, 2003. Proceedings..

[2]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[3]  R. Sekar,et al.  Specification-based anomaly detection: a new approach for detecting network intrusions , 2002, CCS '02.

[4]  Shawn Ostermann,et al.  Detecting Anomalous Network Traffic with Self-organizing Maps , 2003, RAID.

[5]  Charles E. Heckler,et al.  Applied Multivariate Statistical Analysis , 2005, Technometrics.

[6]  Sergio M. Savaresi,et al.  Unsupervised learning techniques for an intrusion detection system , 2004, SAC '04.

[7]  Salvatore J. Stolfo,et al.  Real time data mining-based intrusion detection , 2001, Proceedings DARPA Information Survivability Conference and Exposition II. DISCEX'01.

[8]  Mohammad Zulkernine,et al.  Anomaly Based Network Intrusion Detection with Unsupervised Outlier Detection , 2006, 2006 IEEE International Conference on Communications.

[9]  Young-Seuk Park,et al.  Self-Organizing Map , 2008 .

[10]  Yuehui Chen,et al.  Feature Selection and Intrusion Detection Using Hybrid Flexible Neural Tree , 2005, ISNN.

[11]  Satinder Singh,et al.  Unsupervised Anomaly Detection in Network Intrusion Detection Using Clusters , 2005, ACSC.

[12]  Malcolm I. Heywood,et al.  Selecting Features for Intrusion Detection: A Feature Relevance Analysis on KDD 99 , 2005, PST.

[13]  Anup K. Ghosh,et al.  A Study in Using Neural Networks for Anomaly and Misuse Detection , 1999, USENIX Security Symposium.

[14]  Tao Peng,et al.  Data Mining for Network Intrusion Detection System in Real Time , 2006 .

[15]  David G. Stork,et al.  Pattern Classification , 1973 .

[16]  Li Jun,et al.  HIDE: a Hierarchical Network Intrusion Detection System Using Statistical Preprocessing and Neural Network Classification , 2001 .

[17]  Ali A. Ghorbani,et al.  Research on Intrusion Detection and Response: A Survey , 2005, Int. J. Netw. Secur..

[18]  M. Shyu,et al.  A Novel Anomaly Detection Scheme Based on Principal Component Classifier , 2003 .

[19]  Ajith Abraham,et al.  Feature deduction and ensemble design of intrusion detection systems , 2005, Comput. Secur..

[20]  Salvatore J. Stolfo,et al.  A data mining framework for building intrusion detection models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[21]  Jaideep Srivastava,et al.  A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection , 2003, SDM.

[22]  Salvatore J. Stolfo,et al.  A Geometric Framework for Unsupervised Anomaly Detection , 2002, Applications of Data Mining in Computer Security.

[23]  Leonid Portnoy,et al.  Intrusion detection with unlabeled data using clustering , 2000 .

[24]  Philip K. Chan,et al.  Learning Rules and Clusters for Anomaly Detection in Network Traffic , 2005 .

[25]  H. S. Teng,et al.  Adaptive real-time anomaly detection using inductively generated sequential patterns , 1990, Proceedings. 1990 IEEE Computer Society Symposium on Research in Security and Privacy.

[26]  Malcolm I. Heywood,et al.  A Hierarchical SOM based Intrusion Detection System , 2008 .

[27]  Qiang Chen,et al.  Multivariate Statistical Analysis of Audit Trails for Host-Based Intrusion Detection , 2002, IEEE Trans. Computers.

[28]  Fanyuan Ma,et al.  An unsupervised anomaly detection patterns learning algorithm , 2003, International Conference on Communication Technology Proceedings, 2003. ICCT 2003..

[29]  Eleazar Eskin,et al.  A GEOMETRIC FRAMEWORK FOR UNSUPERVISED ANOMALY DETECTION: DETECTING INTRUSIONS IN UNLABELED DATA , 2002 .

[30]  Qingshan Jiang,et al.  Clustering Ensemble based on the Fuzzy KNN Algorithm , 2007, Eighth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing (SNPD 2007).

[31]  K Raghuveer,et al.  Performance evaluation of data clustering techniques using KDD Cup-99 Intrusion detection data set , 2012 .

[32]  Teuvo Kohonen,et al.  The self-organizing map , 1990, Neurocomputing.