Improving K-means clustering with enhanced Firefly Algorithms

In this research, we propose two variants of the Firefly Algorithm (FA), namely inward intensified exploration FA (IIEFA) and compound intensified exploration FA (CIEFA), for undertaking the obstinate problems of initialization sensitivity and local optima traps of the K-means clustering model. To enhance the capability of both exploitation and exploration, matrix-based search parameters and dispersing mechanisms are incorporated into the two proposed FA models. We first replace the attractiveness coefficient with a randomized control matrix in the IIEFA model to release the FA from the constraints of biological law, as the exploitation capability in the neighbourhood is elevated from a one-dimensional to multi-dimensional search mechanism with enhanced diversity in search scopes, scales, and directions. Besides that, we employ a dispersing mechanism in the second CIEFA model to dispatch fireflies with high similarities to new positions out of the close neighbourhood to perform global exploration. This dispersing mechanism ensures sufficient variance between fireflies in comparison to increase search efficiency. The ALL-IDB2 database, a skin lesion data set, and a total of 15 UCI data sets are employed to evaluate efficiency of the proposed FA models on clustering tasks. The minimum Redundancy Maximum Relevance (mRMR)-based feature selection method is also adopted to reduce feature dimensionality. The empirical results indicate that the proposed FA models demonstrate statistically significant superiority in both distance and performance measures for clustering tasks in comparison with conventional K-means clustering, five classical search methods, and five advanced FA variants.

[1]  Adil Baykasoglu,et al.  Quantum firefly swarms for multimodal dynamic optimization problems , 2019, Expert Syst. Appl..

[2]  Abhijit Banerjee,et al.  Modified firefly algorithm for area estimation and tracking of fast expanding oil spills , 2018, Appl. Soft Comput..

[3]  Vincenzo Piuri,et al.  All-IDB: The acute lymphoblastic leukemia image database for image processing , 2011, 2011 18th IEEE International Conference on Image Processing.

[4]  Xin-She Yang,et al.  Firefly Algorithms for Multimodal Optimization , 2009, SAGA.

[5]  Khaled S. Al-Sultan,et al.  A Tabu search approach to the clustering problem , 1995, Pattern Recognit..

[6]  Magdalene Marinaki,et al.  Ant colony and particle swarm optimization for financial classification problems , 2009, Expert Syst. Appl..

[7]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[8]  Mahdi Yaghoobi,et al.  A new method in multimodal optimization based on firefly algorithm , 2016, Artificial Intelligence Review.

[9]  Li Zhang,et al.  Evolving Ensemble Models for Image Segmentation Using Enhanced Particle Swarm Optimization , 2019, IEEE Access.

[10]  Xin-She Yang,et al.  Firefly algorithm with chaos , 2013, Commun. Nonlinear Sci. Numer. Simul..

[11]  Xianda Zhang,et al.  A genetic algorithm with gene rearrangement for K-means clustering , 2009, Pattern Recognit..

[12]  Ponnuthurai N. Suganthan,et al.  Real-parameter evolutionary multimodal optimization - A survey of the state-of-the-art , 2011, Swarm Evol. Comput..

[13]  Li Zhang,et al.  Intelligent leukaemia diagnosis with bare-bones PSO based feature optimization , 2017, Appl. Soft Comput..

[14]  P. Alam ‘E’ , 2021, Composites Engineering: An A–Z Guide.

[15]  Songwei Huang,et al.  Modified firefly algorithm based multilevel thresholding for color image segmentation , 2017, Neurocomputing.

[16]  Karim Mokrani,et al.  Improved Fuzzy C-Means based Particle Swarm Optimization (PSO) initialization and outlier rejection with level set methods for MR brain image segmentation , 2015, Comput. Methods Programs Biomed..

[17]  Om Prakash Verma,et al.  Opposition and dimensional based modified firefly algorithm , 2016, Expert Syst. Appl..

[18]  Hui Wang,et al.  Firefly algorithm with neighborhood attraction , 2017, Inf. Sci..

[19]  V. Mani,et al.  Clustering using firefly algorithm: Performance study , 2011, Swarm Evol. Comput..

[20]  R. GeethaRamani,et al.  Macula segmentation and fovea localization employing image processing and heuristic based clustering for automated retinal screening , 2018, Comput. Methods Programs Biomed..

[21]  Anabela Afonso,et al.  Overview of Friedman’s Test and Post-hoc Analysis , 2015, Commun. Stat. Simul. Comput..

[22]  P. Alam ‘S’ , 2021, Composites Engineering: An A–Z Guide.

[23]  Mohammad Reza Meybodi,et al.  A new hybrid approach for data clustering using firefly algorithm and K-means , 2012, The 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP 2012).

[24]  A. Gandomi,et al.  Mixed variable structural optimization using Firefly Algorithm , 2011 .

[25]  Nikos A. Vlassis,et al.  The global k-means clustering algorithm , 2003, Pattern Recognit..

[26]  Kenneth Sörensen,et al.  Metaheuristics - the metaphor exposed , 2015, Int. Trans. Oper. Res..

[27]  Ganapati Panda,et al.  A survey on nature inspired metaheuristic algorithms for partitional clustering , 2014, Swarm Evol. Comput..

[28]  Long Quan,et al.  A novel data clustering algorithm based on modified gravitational search algorithm , 2017, Eng. Appl. Artif. Intell..

[29]  Amit Konar,et al.  Synergism of Firefly Algorithm and Q-Learning for Robot Arm Path Planning , 2018, Swarm Evol. Comput..

[30]  Nauman Aslam,et al.  An Intelligent Decision Support System for Leukaemia Diagnosis using Microscopic Blood Images , 2015, Scientific Reports.

[31]  Mei Xie,et al.  MR image segmentation and bias field estimation based on coherent local intensity clustering with total variation regularization , 2016, Medical & Biological Engineering & Computing.

[32]  Aboul Ella Hassanien,et al.  Adaptive k-means clustering algorithm for MR breast image segmentation , 2013, Neural Computing and Applications.

[33]  Patrick D. Surry,et al.  Fitness Variance of Formae and Performance Prediction , 1994, FOGA.

[34]  Li Zhang,et al.  Classifier ensemble reduction using a modified firefly algorithm: An empirical evaluation , 2018, Expert Syst. Appl..

[35]  Dervis Karaboga,et al.  A novel clustering approach: Artificial Bee Colony (ABC) algorithm , 2011, Appl. Soft Comput..

[36]  Xin-She Yang,et al.  Multiobjective firefly algorithm for continuous optimization , 2012, Engineering with Computers.

[37]  M. Friedman The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[38]  Cecilia Di Ruberto,et al.  Leucocyte classification for leukaemia detection using image processing techniques , 2014, Artif. Intell. Medicine.

[39]  Qingmao Hu,et al.  Segmentation of Brain Tissues from Magnetic Resonance Images Using Adaptively Regularized Kernel-Based Fuzzy C-Means Clustering , 2015, Comput. Math. Methods Medicine.

[40]  Anil K. Jain Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[41]  B. Bullnheimer,et al.  A NEW RANK BASED VERSION OF THE ANT SYSTEM: A COMPUTATIONAL STUDY , 1997 .

[42]  Ujjwal Maulik,et al.  Genetic algorithm-based clustering technique , 2000, Pattern Recognit..

[43]  Xin-She Yang,et al.  Firefly Algorithm, Lévy Flights and Global Optimization , 2010, SGAI Conf..

[44]  Ching-Yi Chen,et al.  Particle swarm optimization algorithm and its application to clustering analysis , 2004, 2012 Proceedings of 17th Conference on Electrical Power Distribution.

[45]  Vipin Kumar,et al.  The Challenges of Clustering High Dimensional Data , 2004 .

[46]  Mohammed Azmi Al-Betar,et al.  Text feature selection with a robust weight scheme and dynamic dimension reduction to text document clustering , 2017, Expert Syst. Appl..

[47]  Nadjet Kamel,et al.  A new quantum chaotic cuckoo search algorithm for data clustering , 2018, Expert Syst. Appl..

[48]  Salwani Abdullah,et al.  Hybridizing firefly algorithms with a probabilistic neural network for solving classification problems , 2015, Appl. Soft Comput..

[49]  Kamlesh Mistry,et al.  Intelligent facial emotion recognition using moth-firefly optimization , 2016, Knowl. Based Syst..

[50]  Salwani Abdullah,et al.  A combined approach for clustering based on K-means and gravitational search algorithms , 2012, Swarm Evol. Comput..

[51]  Taher Niknam,et al.  An efficient hybrid approach based on PSO, ACO and k-means for cluster analysis , 2010, Appl. Soft Comput..

[52]  J. Paulo Davim,et al.  Firefly Algorithm , 2019, Optimizing Engineering Problems through Heuristic Techniques.

[53]  Yu Xue,et al.  A hybrid multi-objective firefly algorithm for big data optimization , 2017, Appl. Soft Comput..

[54]  Claire Cardie,et al.  Proceedings of the Eighteenth International Conference on Machine Learning, 2001, p. 577–584. Constrained K-means Clustering with Background Knowledge , 2022 .

[55]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[56]  Kamlesh Mistry,et al.  Feature selection using firefly optimization for classification and regression models , 2018, Decis. Support Syst..

[57]  Li Zhang,et al.  Evolving Image Classification Architectures With Enhanced Particle Swarm Optimisation , 2018, IEEE Access.

[58]  Li-Chun Wang,et al.  Distributed clustering algorithms for data-gathering in wireless mobile sensor networks , 2007, J. Parallel Distributed Comput..

[59]  Adil Baykasoglu,et al.  An improved firefly algorithm for solving dynamic multidimensional knapsack problems , 2014, Expert Syst. Appl..

[60]  Abdolreza Hatamlou,et al.  An efficient hybrid clustering method based on improved cuckoo optimization and modified particle swarm optimization algorithms , 2018, Appl. Soft Comput..

[61]  Abdolreza Hatamlou,et al.  Black hole: A new heuristic optimization approach for data clustering , 2013, Inf. Sci..

[62]  Guojun Gan,et al.  K-means Clustering with Outlier Removal , 2017, Pattern Recognit. Lett..

[63]  Chung-Horng Lung,et al.  A hybrid clustering technique using quantitative and qualitative data for wireless sensor networks , 2015, Ad Hoc Networks.

[64]  Chang Sup Sung,et al.  A tabu-search-based heuristic for clustering , 2000, Pattern Recognit..

[65]  Lirong Qiu,et al.  An efficient hybrid approach based on PSO, ABC and k-means for cluster analysis , 2021, Multimedia Tools and Applications.

[66]  Václav Snásel,et al.  ACO for continuous function optimization: A performance analysis , 2014, 2014 14th International Conference on Intelligent Systems Design and Applications.

[67]  Seyedali Mirjalili,et al.  SCA: A Sine Cosine Algorithm for solving optimization problems , 2016, Knowl. Based Syst..

[68]  Vince Grolmusz,et al.  On dimension reduction of clustering results in structural bioinformatics. , 2014, Biochimica et biophysica acta.

[69]  Shuhao Yu,et al.  A variable step size firefly algorithm for numerical optimization , 2015, Appl. Math. Comput..

[70]  Li Zhang,et al.  Intelligent skin cancer detection using enhanced particle swarm optimization , 2018, Knowl. Based Syst..

[71]  Miao Qi,et al.  A Modified FCM Algorithm for MRI Brain Image Segmentation , 2008, 2008 International Seminar on Future BioMedical Information Engineering.

[72]  Hui Wang,et al.  Firefly algorithm with random attraction , 2016, Int. J. Bio Inspired Comput..

[73]  B. Kulkarni,et al.  An ant colony approach for clustering , 2004 .

[74]  Giri Babu Kande,et al.  Unsupervised Fuzzy Based Vessel Segmentation In Pathological Digital Fundus Images , 2010, Journal of Medical Systems.

[75]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Hassan Abolhassani,et al.  Harmony K-means algorithm for document clustering , 2009, Data Mining and Knowledge Discovery.

[77]  Xin-She Yang,et al.  Nature-Inspired Metaheuristic Algorithms , 2008 .

[78]  Lubo Zhou Improvement of the Firefly-based K-means Clustering Algorithm , 2018 .

[79]  Shokri Z. Selim,et al.  A simulated annealing algorithm for the clustering problem , 1991, Pattern Recognit..

[80]  Chengke Zhou,et al.  Application of K-Means method to pattern recognition in on-line cable partial discharge monitoring , 2013, IEEE Transactions on Dielectrics and Electrical Insulation.

[81]  이현주 Q. , 2005 .

[82]  Alexis Boukouvalas,et al.  What to Do When K-Means Clustering Fails: A Simple yet Principled Alternative Algorithm , 2016, PloS one.

[83]  Farookh Khadeer Hussain,et al.  Support vector regression with chaos-based firefly algorithm for stock market price forecasting , 2013, Appl. Soft Comput..

[84]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[85]  Maoguo Gong,et al.  Fuzzy C-Means Clustering With Local Information and Kernel Metric for Image Segmentation , 2013, IEEE Transactions on Image Processing.

[86]  Marco Dorigo,et al.  Ant colony optimization for continuous domains , 2008, Eur. J. Oper. Res..

[87]  Francisco Herrera,et al.  ROSEFW-RF: The winner algorithm for the ECBDL'14 big data competition: An extremely imbalanced big data bioinformatics problem , 2015, Knowl. Based Syst..

[88]  Shengxiang Yang,et al.  Ant colony optimization with self-adaptive evaporation rate in dynamic environments , 2014, 2014 IEEE Symposium on Computational Intelligence in Dynamic and Uncertain Environments (CIDUE).

[89]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[90]  Li Zhang,et al.  A scattering and repulsive swarm intelligence algorithm for solving global optimization problems , 2018, Knowl. Based Syst..

[91]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[92]  Dushmanta Kumar Das,et al.  A modified Bee Colony Optimization (MBCO) and its hybridization with k-means for an application to data clustering , 2018, Appl. Soft Comput..

[93]  Salwani Abdullah,et al.  Data Clustering Using Big Bang–Big Crunch Algorithm , 2011 .

[94]  Robert B. Fisher,et al.  A Color and Texture Based Hierarchical K-NN Approach to the Classification of Non-melanoma Skin Lesions , 2013 .

[95]  Janez Brest,et al.  A comprehensive review of firefly algorithms , 2013, Swarm Evol. Comput..

[96]  Seyedali Mirjalili,et al.  Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems , 2015, Neural Computing and Applications.

[97]  A. Chitra,et al.  Paraphrase Extraction using fuzzy hierarchical clustering , 2015, Appl. Soft Comput..