Ensemble fuzzy radial basis function neural networks architecture driven with the aid of multi-optimization through clustering techniques and polynomial-based learning

Abstract This study is concerned with the concept of Ensemble Fuzzy Radial Basis Function Neural Networks (EFRBFNN) endowed with clustering techniques, polynomial-based LSE/WLSE learning as well as their optimization implemented by Multi-Objective Particle Swarm Optimization (MOPSO). The network architecture of the proposed EFRBFNN classifier is considered three types of clustering techniques and four forms of regression polynomials. The proposed classifier can not only fully capture the local distribution (feature) information contained in the data, but also choose effective network architecture for improving classification performance and compactness. Three classes of clustering techniques (such as iterative self-organizing data analysis techniques algorithm (ISODATA), affinity propagation (AP), and extended K-means) are considered to effectively produce membership degrees based on feature information extraction suitable to fit data characteristics. Least Square Error Estimation (LSE) and Weighted Least Square Error Estimation (WLSE)-based learning with four types of regression polynomials are considered to estimate to coefficients of polynomials. MOPSO is exploited for choosing the effective architecture among different clustering techniques, polynomials, and polynomial-based learning with multi-objective parametric optimization. MOPSO helps achieve a sound compromise between the preferred classification performance and the compactness realized with the aid of four objective functions such as a) classification accuracy (CA), b) complexity of clustering, c) complexity of polynomial and d) sum of squared coefficients (SSC). The improvement and effectiveness of the proposed network architecture are quantified on a basis of the comprehensive experimental results and also a comparative analysis is offered to demonstrate the superiority of the proposed classifier.

[1]  Ronghua Shang,et al.  Sparse and low-redundant subspace learning-based dual-graph regularized robust feature selection , 2020, Knowl. Based Syst..

[2]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[3]  C. Coello,et al.  Multiobjective optimization using a micro-genetic algorithm , 2001 .

[4]  M. C. Pegalajar,et al.  A Munsell colour-based approach for soil classification using Fuzzy Logic and Artificial Neural Networks , 2020, Fuzzy Sets Syst..

[5]  Naftali Tishby,et al.  An Information Theoretic Tradeoff between Complexity and Accuracy , 2003, COLT.

[6]  Sung-Kwun Oh,et al.  Hybrid optimization of information granulation-based fuzzy radial basis function neural networks , 2010, Int. J. Intell. Comput. Cybern..

[7]  Sung-Kwun Oh,et al.  Fuzzy quasi-linear SVM classifier: Design and analysis , 2020, Fuzzy Sets Syst..

[8]  Sung-Kwun Oh,et al.  Design of K-means clustering-based polynomial radial basis function neural networks (pRBF NNs) realized with the aid of particle swarm optimization and differential evolution , 2012, Neurocomputing.

[9]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[10]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[11]  Sung-Kwun Oh,et al.  Design of Reinforced Fuzzy Radial Basis Function Neural Network Classifier Driven With the Aid of Iterative Learning Techniques and Support Vector-Based Clustering , 2021, IEEE Transactions on Fuzzy Systems.

[12]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[13]  Sung-Kwun Oh,et al.  Polynomial-based radial basis function neural networks (P-RBF NNs) realized with the aid of particle swarm optimization , 2011, Fuzzy Sets Syst..

[14]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[15]  Limin Fu,et al.  FLAME, a novel fuzzy clustering method for the analysis of DNA microarray data , 2007, BMC Bioinformatics.

[16]  Sung-Kwun Oh,et al.  Multiobjective Space Search Optimization and Information Granulation in the Design of Fuzzy Radial Basis Function Neural Networks , 2012 .

[17]  Ronghua Shang,et al.  Local discriminative based sparse subspace learning for feature selection , 2019, Pattern Recognit..

[18]  Byoung-Jun Park,et al.  Analytic design of information granulation-based fuzzy radial basis function neural networks with the aid of multiobjective particle swarm optimization , 2012, Int. J. Intell. Comput. Cybern..

[19]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[20]  Hao Yu,et al.  Advantages of Radial Basis Function Networks for Dynamic System Design , 2011, IEEE Transactions on Industrial Electronics.

[21]  M. N. Vrahatis,et al.  Particle swarm optimization method in multiobjective problems , 2002, SAC '02.

[22]  Jinglu Hu,et al.  Multi‐SVM classifier system with piecewise interpolation , 2013 .

[23]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[24]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[25]  Shuai Jiang,et al.  A Simple and Fast Algorithm for Global K-means Clustering , 2010, 2010 Second International Workshop on Education Technology and Computer Science.