An effective unsupervised network anomaly detection method

In this paper, we present an effective tree based subspace clustering technique (TreeCLUS) for finding clusters in network intrusion data and for detecting unknown attacks without using any labelled traffic or signatures or training. To establish its effectiveness in finding all possible clusters, we perform a cluster stability analysis. We also introduce an effective cluster labelling technique (CLUSLab) to generate labelled dataset based on the stable cluster set generated by TreeCLUS. CLUSLab is a multi-objective technique that exploits an ensemble approach for stability analysis of the clusters generated by TreeCLUS. We evaluate the performance of both TreeCLUS and CLUSLab in terms of several real world intrusion datasets to identify unknown attacks and find that both outperform the competing algorithms.

[1]  Joachim M. Buhmann,et al.  Stability-Based Validation of Clustering Solutions , 2004, Neural Computation.

[2]  J. Dunn.  Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[3]  Sabu M. Thampi,et al.  Proceedings of the International Conference on Advances in Computing, Communications and Informatics , 2012 .

[4]  P. Rousseeuw.  Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[5]  Gerardo Beni,et al.  A Validity Measure for Fuzzy Clustering , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Sunghae Jun,et al.  An Ensemble Method for Validation of Cluster Analysis , 2011 .

[7]  Satinder Singh,et al.  Unsupervised Anomaly Detection in Network Intrusion Detection Using Clusters , 2005, ACSC.

[8]  L. Hubert,et al.  Quadratic assignment as a general data analysis strategy. , 1976 .

[9]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.

[10]  Nasser Yazdani,et al.  Mutual information-based feature selection for intrusion detection systems , 2011, J. Netw. Comput. Appl..

[11]  Leonid Portnoy,et al.  Intrusion detection with unlabeled data using clustering , 2000 .

[12]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Chih-Fong Tsai,et al.  A triangle area based nearest neighbors approach to intrusion detection , 2010, Pattern Recognit..

[14]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[15]  Ming-Yang Su,et al.  Using clustering to improve the KNN-based classifiers for online anomaly network traffic identification , 2011, J. Netw. Comput. Appl..

[16]  Shi-Jinn Horng,et al.  A novel intrusion detection system based on hierarchical clustering and support vector machines , 2011, Expert Syst. Appl..

[17]  Hiroki Takakura,et al.  Toward a more practical unsupervised anomaly detection system , 2013, Inf. Sci..

[18]  Shai Ben-David,et al.  A Sober Look at Clustering Stability , 2006, COLT.

[19]  Philippe Owezarski,et al.  Unsupervised Network Intrusion Detection Systems: Detecting the Unknown without Knowledge , 2012, Comput. Commun..

[20]  Rachid Beghdad,et al.  Critical Study of Supervised Learning Techniques in Predicting Attacks , 2010, Inf. Secur. J. A Glob. Perspect..

[21]  Jugal K. Kalita,et al.  NADO: network anomaly detection using outlier approach , 2011, ICCCS '11.

[22]  Adel Nadjaran Toosi,et al.  A new approach to intrusion detection based on an evolutionary soft computing model using neuro-fuzzy classifiers , 2007, Comput. Commun..

[23]  Hui Wang,et al.  A clustering-based method for unsupervised intrusion detections , 2006, Pattern Recognit. Lett..

[24]  Michalis Vazirgiannis,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques , 2022 .

[25]  Ali A. Ghorbani,et al.  IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART C: APPLICATIONS AND REVIEWS 1 Toward Credible Evaluation of Anomaly-Based Intrusion-Detection Methods , 2022 .