PSO+K-means Algorithm for Anomaly Detection in Big Data

The use of clustering methods in anomaly detection is considered as an effective approach. The choice of the cluster primary center and the finding of local optimum in the well-known k-means and other classic clustering algorithms are considered as one of the major problems and do not allow to get accurate results in anomaly detection. In this paper to improve the accuracy of anomaly detection based on the combination of PSO (particle swarm optimization) and k-means algorithms, the new weighted clustering method is proposed. The proposed method is tested on Yahoo! S5 dataset and a comparative analysis of the obtained results with the k-means algorithm is performed. The results of experiments show that compared to the k-means algorithm the proposed method is more robust and allows to get more accurate results.

[1]  Lizhong Xiao,et al.  K-means Algorithm Based on Particle Swarm Optimization Algorithm for Anomaly Intrusion Detection , 2006, 2006 6th World Congress on Intelligent Control and Automation.

[2]  Eloı́sa Macedo.  Two-Step Semidefinite Programming approach to clustering and dimensionality reduction , 2015 .

[3]  Rajesh Kumar,et al.  A boundary restricted adaptive particle swarm optimization for data clustering , 2013, Int. J. Mach. Learn. Cybern..

[4]  David C. Yen,et al.  A Network Behavior-Based Botnet Detection Mechanism Using PSO and K-means , 2015, TMIS.

[5]  J. Dunn.  Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[6]  Jinfeng Han,et al.  The Clustering Algorithm Based on Particle Swarm Optimization Algorithm , 2008, 2008 International Conference on Intelligent Computation Technology and Automation (ICICTA).

[7]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[8]  Can Atilgan,et al.  A Space Efficient Minimum Spanning Tree Approach to the Fuzzy Joint Points Clustering Algorithm , 2019, IEEE Transactions on Fuzzy Systems.

[9]  Sharandeep Singh.  A Review on Particle Swarm Optimization Algorithm , 2014 .

[10]  Stan Matwin,et al.  A review on particle swarm optimization algorithm and its variants to clustering high-dimensional data , 2013, Artificial Intelligence Review.

[11]  Adil M. Bagirov,et al.  New diagonal bundle method for clustering problems in large data sets , 2017, Eur. J. Oper. Res..

[12]  Ramiz M. Aliguliyev,et al.  Performance evaluation of density-based clustering methods , 2009, Inf. Sci..

[13]  Efendi N. Nasibov,et al.  A Note on Fuzzy Joint Points Clustering Methods for Large Datasets , 2016, IEEE Transactions on Fuzzy Systems.

[14]  Junyan Chen,et al.  HYBRID CLUSTERING ALGORITHM BASED ON PSO WITH THE MULTIDIMENSIONAL ASYNCHRONISM AND STOCHASTIC DISTURBANCE METHOD , 2012 .

[15]  Yongzhong Li,et al.  Anomaly Intrusion Detection Method Based on K-Means Clustering Algorithm with Particle Swarm Optimization , 2011, 2011 International Conference of Information Technology, Computer Engineering and Management Sciences.

[16]  Ian F. C. Smith,et al.  A Bounded Index for Cluster Validity , 2007, MLDM.

[17]  R. J. Kuo,et al.  An application of particle swarm optimization algorithm to clustering analysis , 2011, Soft Comput..

[18]  Georgios Kambourakis,et al.  Swarm intelligence in intrusion detection: A survey , 2011, Comput. Secur..

[19]  Adil M. Bagirov,et al.  Clustering in large data sets with the limited memory bundle method , 2018, Pattern Recognit..

[20]  Rasim M. Alguliyev,et al.  Weighted Clustering for Anomaly Detection in Big Data , 2018, Statistics, Optimization & Information Computing.

[21]  Adil Bagirov,et al.  Batch clustering algorithm for big data sets , 2016, 2016 IEEE 10th International Conference on Application of Information and Communication Technologies (AICT).