Hybridization of magnetic charge system search and particle swarm optimization for efficient data clustering using neighborhood search strategy

Clustering is a popular data analysis technique, which is applied for partitioning of datasets. The aim of clustering is to arrange the data items into clusters based on the values of their attributes. Magnetic charge system search (MCSS) algorithm is a new meta-heuristic optimization algorithm inspired by the electromagnetic theory. It has been proved better than other meta-heuristics. This paper presents a new hybrid meta-heuristic algorithm by combining both MCSS and particle swarm optimization (PSO) algorithms, which is called MCSS–PSO, for partitional clustering problem. Moreover, a neighborhood search strategy is also incorporated in this algorithm to generate more promising solutions. The performance of the proposed MCSS–PSO algorithm is tested on several benchmark datasets and its performance is compared with already existing clustering algorithms such as K-means, PSO, genetic algorithm, ant colony optimization, charge system search, chaotic charge system search algorithm, and some PSO variants. From the experimental results, it can be seen that performance of the proposed algorithm is better than the other algorithms being compared and it can be effectively used for partitional clustering problem.

[1]  Pekka Teppola,et al.  Adaptive Fuzzy C-Means clustering in process monitoring , 1999 .

[2]  W. Pan,et al.  Cluster analysis using multivariate normal mixture models to detect differential gene expression with microarray data , 2006, Comput. Stat. Data Anal..

[3]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[4]  Cheng-Fa Tsai,et al.  ACODF: a novel data clustering approach for data mining in large databases , 2004 .

[5]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[6]  Shokri Z. Selim,et al.  A simulated annealing algorithm for the clustering problem , 1991, Pattern Recognit..

[7]  Abdolreza Hatamlou,et al.  Black hole: A new heuristic optimization approach for data clustering , 2013, Inf. Sci..

[8]  Jing Li,et al.  Ant clustering algorithm with K-harmonic means clustering , 2010, Expert Syst. Appl..

[9]  Gadadhar Sahoo,et al.  A charged system search approach for data clustering , 2014, Progress in Artificial Intelligence.

[10]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[11]  A. Kaveh,et al.  An improved magnetic charged system search for optimization of truss structures with continuous and discrete variables , 2015, Appl. Soft Comput..

[12]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[13]  Chih Chieh Yang,et al.  Integration of Ant Colony SOM and K-Means for Clustering Analysis , 2006, KES.

[14]  Dorothy Ndedi Monekosso,et al.  A review of ant algorithms , 2009, Expert Syst. Appl..

[15]  M.H. Hassoun,et al.  Fundamentals of Artificial Neural Networks , 1996, Proceedings of the IEEE.

[16]  Jiong Yang,et al.  STING: A Statistical Information Grid Approach to Spatial Data Mining , 1997, VLDB.

[17]  David L. Robertson,et al.  CTree: comparison of clusters between phylogenetic trees made easy , 2007, Bioinform..

[18]  Gadadhar Sahoo,et al.  Ant colony based hybrid optimization for data clustering , 2007, Kybernetes.

[19]  Erwie Zahara,et al.  A hybridized approach to data clustering , 2008, Expert Syst. Appl..

[20]  Lingling Huang,et al.  A novel artificial bee colony algorithm with Powell's method , 2013, Appl. Soft Comput..

[21]  Dervis Karaboga,et al.  A novel clustering approach: Artificial Bee Colony (ABC) algorithm , 2011, Appl. Soft Comput..

[22]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[23]  Francisco Herrera,et al.  Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power , 2010, Inf. Sci..

[24]  W. J. Dunn,et al.  USE OF CLUSTER ANALYSIS IN THE DEVELOPMENT OF STRUCTURE-ACTIVITY RELATIONS FOR ANTITUMOR TRIAZENES , 1977 .

[25]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[26]  T. B. Murphy,et al.  Gaussian Parsimonious Clustering Models with Covariates , 2017 .

[27]  A. Kaveh,et al.  A novel hybrid charge system search and particle swarm optimization method for multi-objective optimization , 2011, Expert Syst. Appl..

[28]  C. A. Murthy,et al.  In search of optimal clusters using genetic algorithms , 1996, Pattern Recognit. Lett..

[29]  Sudipto Guha,et al.  CURE: an efficient clustering algorithm for large databases , 1998, SIGMOD '98.

[30]  Hong Zhou,et al.  Accurate integration of multi-view range images using k-means clustering , 2008, Pattern Recognit..

[31]  James C. Bezdek,et al.  Genetic algorithm guided clustering , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[32]  Jing Li,et al.  A new hybrid method based on partitioning-based DBSCAN and ant clustering , 2011, Expert Syst. Appl..

[33]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[34]  R. Tibshirani,et al.  An introduction to the bootstrap , 1993 .

[35]  Hossein Nezamabadi-pour,et al.  Disruption: A new operator in gravitational search algorithm , 2011, Sci. Iran..

[36]  Ujjwal Maulik,et al.  Genetic algorithm-based clustering technique , 2000, Pattern Recognit..

[37]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[38]  B. Kulkarni,et al.  An ant colony approach for clustering , 2004 .

[39]  P. Sneath.  The application of computers to taxonomy. , 1957, Journal of general microbiology.

[40]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[41]  R. J. Kuo,et al.  Application of a hybrid of genetic algorithm and particle swarm optimization algorithm for order clustering , 2010, Decis. Support Syst..

[42]  Hans-Peter Kriegel,et al.  OPTICS: ordering points to identify the clustering structure , 1999, SIGMOD '99.

[43]  Ching-Yi Chen,et al.  Particle swarm optimization algorithm and its application to clustering analysis , 2004, 2012 Proceedings of 17th Conference on Electrical Power Distribution.

[44]  Anima Naik,et al.  Data Clustering Based on Teaching-Learning-Based Optimization , 2011, SEMCCO.

[45]  Gérard Govaert,et al.  Gaussian parsimonious clustering models , 1995, Pattern Recognit..

[46]  A. Kaveh,et al.  A novel heuristic optimization method: charged system search , 2010 .

[47]  Budi Santosa,et al.  Cat Swarm Optimization for Clustering , 2009, 2009 International Conference of Soft Computing and Pattern Recognition.

[48]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[49]  Tian Zhang,et al.  BIRCH: an efficient data clustering method for very large databases , 1996, SIGMOD '96.

[50]  Alex Alves Freitas,et al.  A Survey of Evolutionary Algorithms for Clustering , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[51]  Robert R. Sokal,et al.  A statistical method for evaluating systematic relationships , 1958 .

[52]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[53]  Kevin Cheng,et al.  An ACO-Based Clustering Algorithm , 2006, ANTS Workshop.

[54]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[55]  Adrian E. Raftery,et al.  MCLUST: Software for Model-Based Cluster Analysis , 1999 .

[56]  R. J. Kuo,et al.  Application of ant K-means on clustering analysis , 2005 .

[57]  Lin-Yu Tseng,et al.  Genetic algorithms for clustering, feature selection and classification , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[58]  Bernard De Baets,et al.  UPGMA clustering revisited: A weight-driven approach to transitive approximation , 2006, Int. J. Approx. Reason..

[59]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[60]  Ajith Abraham,et al.  Swarm Intelligence Algorithms for Data Clustering , 2008, Soft Computing for Knowledge Discovery and Data Mining.

[61]  A. Kaveh,et al.  Magnetic charged system search: a new meta-heuristic algorithm for optimization , 2012, Acta Mechanica.

[62]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[63]  Chang Sup Sung,et al.  A tabu-search-based heuristic for clustering , 2000, Pattern Recognit..

[64]  Ibrahim Eksin,et al.  A new optimization method: Big Bang-Big Crunch , 2006, Adv. Eng. Softw..

[65]  R. Venkata Rao,et al.  Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems , 2011, Comput. Aided Des..

[66]  Yi Lu,et al.  FGKA: a Fast Genetic K-means Clustering Algorithm , 2004, SAC '04.

[67]  S..  Nonlinear time series modelling and prediction using Gaussian RBF networks with enhanced clustering and RLS learning , 2004 .

[68]  Sabu M. Thampi,et al.  Advances in Signal Processing and Intelligent Recognition Systems - selected and revised papers from the International Symposium on Signal Processing and Intelligent Recognition Systems, SIRS 2014, March 13-15, 2014, Trivandrum, India , 2014, SIRS.

[69]  H. Edelsbrunner,et al.  Efficient algorithms for agglomerative hierarchical clustering methods , 1984 .

[70]  Gadadhar Sahoo,et al.  An Improved Cat Swarm Optimization Algorithm for Clustering , 2015 .

[71]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[72]  Sudipto Guha,et al.  ROCK: a robust clustering algorithm for categorical attributes , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[73]  Lin-Yu Tseng,et al.  A genetic approach to the automatic clustering problem , 2001, Pattern Recognit..

[74]  Ian Davidson,et al.  Constrained Clustering: Advances in Algorithms, Theory, and Applications , 2008 .

[75]  Georgios C. Anagnostopoulos,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2003, Lecture Notes in Computer Science.

[76]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[77]  M. Narasimha Murty,et al.  Genetic K-means algorithm , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[78]  Ujjwal Maulik,et al.  Simulated annealing based automatic fuzzy clustering combined with ANN classification for analyzing microarray data , 2010, Comput. Oper. Res..

[79]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[80]  T. Sørensen,et al.  A method of establishing group of equal amplitude in plant sociobiology based on similarity of species content and its application to analyses of the vegetation on Danish commons , 1948 .

[81]  Angelo Dalli.  Adaptation of the F-measure to Cluster Based Lexicon Quality Evaluation , 2003 .

[82]  Gadadhar Sahoo,et al.  A Chaotic Charged System Search Approach for Data Clustering , 2014, Informatica.

[83]  Thomas Stützle,et al.  Ant Colony Optimization and Swarm Intelligence , 2008 .

[84]  Xiaohui Yan,et al.  A new approach for data clustering using hybrid artificial bee colony algorithm , 2012, Neurocomputing.

[85]  Mohamed S. Kamel,et al.  An aggregated clustering approach using multi-ant colonies algorithms , 2006, Pattern Recognit..

[86]  Ethem Alpaydin,et al.  Introduction to machine learning , 2004, Adaptive computation and machine learning.

[87]  Michael R. Anderberg,et al.  Cluster Analysis for Applications , 1973 .

[88]  Adam P. Arkin,et al.  FastTree: Computing Large Minimum Evolution Trees with Profiles instead of a Distance Matrix , 2009, Molecular biology and evolution.

[89]  Lakhmi C. Jain,et al.  Knowledge-Based Intelligent Information and Engineering Systems , 2004, Lecture Notes in Computer Science.

[90]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[91]  H. Altay Güvenir,et al.  Classification by Voting Feature Intervals , 1997, ECML.

[92]  Léon Bottou,et al.  Local Learning Algorithms , 1992, Neural Computation.

[93]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[94]  Gal Chechik,et al.  Information Bottleneck for Gaussian Variables , 2003, J. Mach. Learn. Res..

[95]  Andrew R. Webb,et al.  Statistical Pattern Recognition , 1999 .

[96]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[97]  Yugal Kumar,et al.  Modified Teacher Learning Based Optimization Method for Data Clustering , 2014, SIRS.

[98]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[99]  Taher Niknam,et al.  An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis , 2010, Appl. Soft Comput..

[100]  Marco Dorigo,et al.  On the Performance of Ant-based Clustering , 2003, HIS.

[101]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[102]  Stan Matwin,et al.  Data Clustering Using Hybrid Particle Swarm Optimization , 2012, IDEAL.

[103]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[104]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[105]  Geoffrey I. Webb,et al.  MultiBoosting: A Technique for Combining Boosting and Wagging , 2000, Machine Learning.

[106]  Kuang Yu Huang,et al.  Author ' s personal copy A hybrid particle swarm optimization approach for clustering and classification of datasets , 2011 .