Learning dynamic simultaneous clustering and classification via automatic differential evolution and firework algorithm

Abstract Semi-supervised learning is significant data analysis method in the age of Big Data. The Bayesian-based classifier is a classical classification method in semi-supervised learning. Wherein, the classifier with clustering and classification technology have experienced the transformation from sequential structure to simultaneous structure. There are two main difficulties in simultaneous structure: the limited accuracy and diversity caused by rigid optimization algorithm; and the imbalance status between clustering and classification processes caused by insufficiently structure. To overcome these difficulties, a novel multi-objective differential evolution and firework algorithm for automatic simultaneous clustering and classification algorithm (MASCC-DE/FWA) is proposed. The main contributions of MASCC-DE/FWA contain: (1) Combination searching strategy for dynamic searching (2) Rapid and low-complexity silhouette coefficient as redesigned clustering objective function (3) Automatic clustering, opposition-based learning and adjusted mutual information for strengthening SCC-MOEA framework. The experimental result demonstrates that MASCC-DE/FWA performs better than other 8 state-of-art classification algorithms on synthetic dataset, 19 UCI datasets and image segmentation tasks.

[1]  Eckart Zitzler,et al.  Indicator-Based Selection in Multiobjective Search , 2004, PPSN.

[2]  Swagatam Das,et al.  Automatic Clustering Using an Improved Differential Evolution Algorithm , 2007 .

[3]  Alvaro Garcia-Piquer,et al.  Large-Scale Experimental Evaluation of Cluster Representations for Multiobjective Evolutionary Clustering , 2014, IEEE Transactions on Evolutionary Computation.

[4]  A. Ferligoj,et al.  Direct multicriteria clustering algorithms , 1992 .

[5]  Fazhi He,et al.  3D mesh simplification with feature preservation based on Whale Optimization Algorithm and Differential Evolution , 2020, Integr. Comput. Aided Eng..

[6]  Yangyang Li,et al.  A particle swarm optimization based simultaneous learning framework for clustering and classification , 2014, Pattern Recognit..

[7]  Ujjwal Maulik,et al.  Validity index for crisp and fuzzy clusters , 2004, Pattern Recognit..

[8]  João Miguel da Costa Sousa,et al.  Modified binary PSO for feature selection using SVM applied to mortality prediction of septic patients , 2013, Appl. Soft Comput..

[9]  K. Thangavel,et al.  Clustering Categorical Data Using Silhouette Coefficient as a Relocating Measure , 2007, International Conference on Computational Intelligence and Multimedia Applications (ICCIMA 2007).

[10]  Yilin Chen,et al.  A new haze removal approach for sky/river alike scenes based on external and internal clues , 2019, Multimedia Tools and Applications.

[11]  Daoqiang Zhang,et al.  A Multiobjective Simultaneous Learning Framework for Clustering and Classification , 2010, IEEE Transactions on Neural Networks.

[12]  Quan Quan,et al.  A multi-phase blending method with incremental intensity for training detection networks , 2020, The Visual Computer.

[13]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[14]  Dejun Zhang,et al.  A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration , 2020, Appl. Soft Comput..

[15]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[16]  Haoran Li,et al.  Multi-objective self-organizing optimization for constrained sparse array synthesis , 2020, Swarm Evol. Comput..

[17]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[18]  Y. Ho,et al.  Simple Explanation of the No-Free-Lunch Theorem and Its Implications , 2002 .

[19]  Joshua D. Knowles,et al.  An Evolutionary Approach to Multiobjective Clustering , 2007, IEEE Transactions on Evolutionary Computation.

[20]  M. Matsen.  The standard Gaussian model for block copolymer melts , 2002 .

[21]  Yen-Jen Oyang,et al.  Data classification with radial basis function networks based on a novel kernel density estimation algorithm , 2005, IEEE Transactions on Neural Networks.

[22]  Qiang Qian,et al.  Simultaneous clustering and classification over cluster structure representation , 2012, Pattern Recognit..

[23]  Benjamin H. Good,et al.  Performance of modularity maximization in practical contexts. , 2009, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  Meng Zhao,et al.  Very large-scale data classification based on K-means clustering and multi-kernel SVM , 2019, Soft Comput..

[25]  Jin Liu and Tuan D. Pham.  Fuzzy Clustering for Microarray Data Analysis: A Review , 2011 .

[26]  Joshua D. Knowles,et al.  A New Reduced-Length Genetic Representation for Evolutionary Multiobjective Clustering , 2017, EMO.

[27]  Gianluca Tempesti,et al.  A multiobjective metaheuristic approach for morphological filters on many-core architectures , 2019, Integr. Comput. Aided Eng..

[28]  Qingfu Zhang,et al.  Efficient Nondomination Level Update Method for Steady-State Evolutionary Multiobjective Optimization , 2017, IEEE Transactions on Cybernetics.

[29]  Sultan Noman Qasem,et al.  Multi-objective hybrid evolutionary algorithms for radial basis function neural network design , 2012, Knowl. Based Syst..

[30]  Daoqiang Zhang,et al.  A simultaneous learning framework for clustering and classification , 2009, Pattern Recognit..

[31]  H. Khanna Nehemiah,et al.  Neural network classifier optimization using Differential Evolution with Global Information and Back Propagation algorithm for clinical datasets , 2016, Appl. Soft Comput..

[32]  Yanru Wang,et al.  Research Progress on Semi-Supervised Clustering , 2019, Cognitive Computation.

[33]  Abraham Otero,et al.  A novel and simple strategy for evolving prototype based clustering , 2018, Pattern Recognit..

[34]  Jorge Puente,et al.  Satisfying flexible due dates in fuzzy job shop by means of hybrid evolutionary algorithms , 2018, Integr. Comput. Aided Eng..

[35]  Min Chen,et al.  Rough Cluster Quality Index Based on Decision Theory , 2009, IEEE Transactions on Knowledge and Data Engineering.

[36]  Ludmila I. Kuncheva,et al.  Clustering-and-selection model for classifier combination , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[37]  Yoko Ishino.  Knowledge extraction from web-based consumer surveys: Bayesian networks with feature selection , 2017, Int. J. Web Eng. Technol..

[38]  Shengdong Zhang,et al.  NLDN: Non-local dehazing network for dense haze removal , 2020, Neurocomputing.

[39]  Hossein Karami,et al.  A novel framework to generate clustering algorithms based on a particular classification structure , 2017, 2017 Artificial Intelligence and Signal Processing Conference (AISP).

[40]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[41]  Yung C. Shin,et al.  Radial basis function neural network for approximation and estimation of nonlinear stochastic dynamic systems , 1994, IEEE Trans. Neural Networks.

[42]  Aruna Tiwari,et al.  An improved multiobjective simultaneous learning framework for designing a classifier , 2011, 2011 International Conference on Recent Trends in Information Technology (ICRTIT).

[43]  Oscar Castillo,et al.  Fuzzy granular gravitational clustering algorithm for multivariate data , 2014, Inf. Sci..

[44]  Fazhi He,et al.  Service-Oriented Feature-Based Data Exchange for Cloud-Based Design and Manufacturing , 2018, IEEE Transactions on Services Computing.

[45]  Ronghua Shang,et al.  A novel location-based DNA matching algorithm for hyperspectral image classification , 2019, Memetic Comput..

[46]  Yi Zhou,et al.  Dynamic strategy based parallel ant colony optimization on GPUs for TSPs , 2017, Science China Information Sciences.

[47]  Yu Xue,et al.  A hybrid multi-objective firefly algorithm for big data optimization , 2017, Appl. Soft Comput..

[48]  Hui Wang,et al.  Gaussian Bare-Bones Differential Evolution , 2013, IEEE Transactions on Cybernetics.

[49]  Estevam R. Hruschka,et al.  Combining Classification and Clustering for Tweet Sentiment Analysis , 2014, 2014 Brazilian Conference on Intelligent Systems.

[50]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[51]  Yiteng Pan,et al.  Learning social representations with deep autoencoder for recommender system , 2020, World Wide Web.

[52]  Yongjun Zhang,et al.  An optimizer ensemble algorithm and its application to image registration , 2019, Integr. Comput. Aided Eng..

[53]  Fazhi He,et al.  An efficient and robust bat algorithm with fusion of opposition-based learning and whale optimization algorithm , 2020, Intell. Data Anal..

[54]  Xiao-Liang Shen,et al.  A hybrid particle swarm optimization algorithm using adaptive learning strategy , 2018, Inf. Sci..

[55]  Fuwen Yang,et al.  Robust H/sub /spl infin// filtering for stochastic time-delay systems with missing measurements , 2006, IEEE Transactions on Signal Processing.

[56]  Sultan Noman Qasem,et al.  Memetic Elitist Pareto Differential Evolution algorithm based Radial Basis Function Networks for classification problems , 2011, Appl. Soft Comput..

[57]  Mehrdad Jalali,et al.  An optimized approach for unbalanced big data categorizing using fuzzy clustering , 2014, 2014 International Congress on Technology, Communication and Knowledge (ICTCK).

[58]  Mehdi Ghatee,et al.  NSGA2PI, the hybrid algorithm for Radial Basis function networks , 2013, 2013 Second International Conference on Informatics & Applications (ICIA).

[59]  Joydeep Ghosh,et al.  An Optimization Framework for Combining Ensembles of Classifiers and Clusterers with Applications to Nontransductive Semisupervised Learning and Transfer Learning , 2014, TKDD.

[60]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[61]  Siti Zaiton Mohd Hashim,et al.  Memetic multiobjective particle swarm optimization-based radial basis function network for classification problems , 2013, Inf. Sci..

[62]  Fang Liu,et al.  3D fast convex-hull-based evolutionary multiobjective optimization algorithm , 2018, Appl. Soft Comput..

[63]  Fang Liu,et al.  Learning simultaneous adaptive clustering and classification via MOEA , 2016, Pattern Recognit..

[64]  Dong-Jo Park,et al.  A Novel Validity Index for Determination of the Optimal Number of Clusters , 2001 .

[65]  Licheng Jiao,et al.  A Sparse Spectral Clustering Framework via Multiobjective Evolutionary Algorithm , 2016, IEEE Transactions on Evolutionary Computation.

[66]  Shiwei Wei,et al.  A convergence-diversity balanced fitness evaluation mechanism for decomposition- based many-objective optimization algorithm , 2019, Integr. Comput. Aided Eng..

[67]  Yi Zhou,et al.  An efficient GPU-based parallel tabu search algorithm for hardware/software co-design , 2020, Frontiers of Computer Science.

[68]  Yiteng Pan,et al.  A scalable region-based level set method using adaptive bilateral filter for noisy image segmentation , 2019, Multimedia Tools and Applications.

[69]  Germain Forestier,et al.  Semi-supervised learning using multiple clusterings with limited labeled data , 2016, Inf. Sci..

[70]  Magne Setnes,et al.  Fuzzy relational classifier trained by fuzzy clustering , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[71]  Alvaro Garcia-Piquer,et al.  Analysis of vulnerability assessment results based on CAOS , 2011, Appl. Soft Comput..

[72]  Shingo Mabu,et al.  A sequential subspace face recognition framework using genetic-based clustering , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[73]  Fang Liu,et al.  Artificial immune multi-objective SAR image segmentation with fused complementary features , 2011, Inf. Sci..

[74]  Zhihua Cui,et al.  Improving artificial Bee colony algorithm using a new neighborhood selection mechanism , 2020, Inf. Sci..

[75]  Saurabh Mishra,et al.  Design and Implementation of a Classifier based on Multiobjective Learning Framework with Fuzzy Clustering , 2013 .

[76]  Safia Abdelmounaime,et al.  New Brodatz-Based Image Databases for Grayscale Color and Multiband Texture Analysis , 2013 .

[77]  Fazhi He,et al.  A dividing-based many-objective evolutionary algorithm for large-scale feature selection , 2019, Soft Computing.

[78]  Biao Hou,et al.  Target-Oriented High-Resolution SAR Image Formation via Semantic Information Guided Regularizations , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[79]  Xiao-Liang Shen,et al.  An adaptive weight vector guided evolutionary algorithm for preference-based multi-objective optimization , 2019, Swarm Evol. Comput..

[80]  Qingfu Zhang,et al.  MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition , 2007, IEEE Transactions on Evolutionary Computation.

[81]  Fang Liu,et al.  MOEA/D with opposition-based learning for multiobjective optimization problem , 2014, Neurocomputing.

[82]  Peng Hao,et al.  An adaptive response surface method and Gaussian global-best harmony search algorithm for optimization of aircraft stiffened panels , 2018, Appl. Soft Comput..

[83]  Chih-Chiang Lin,et al.  A New Binary Classifier: Clustering-Launched Classification , 2006, ICIC.

[84]  Shuang Wang,et al.  Fuzzy Superpixels for Polarimetric SAR Images Classification , 2018, IEEE Transactions on Fuzzy Systems.

[85]  Fazhi He,et al.  DRCDN: learning deep residual convolutional dehazing networks , 2019, The Visual Computer.

[86]  Honghai Wang,et al.  A multi-objective differential evolutionary algorithm for constrained multi-objective optimization problems with low feasible ratio , 2019, Appl. Soft Comput..

[87]  Ying Tan,et al.  Fireworks Algorithm for Optimization , 2010, ICSI.