A hybrid classification algorithm based on coevolutionary EBFNN and domain covering method

A new hybrid scheme of the elliptical basis function neural network (EBFNN) model combined with the cooperative coevolutionary algorithm (Co-CEA) and domain covering method is presented for multiclass classification tasks. This combination of the Co-CEA EBFNN (CC-EBFNN) and the domain covering method is proposed to enhance the predictive capability of the estimated model. The whole training process is divided into two stages: the evolutionary process, and the heuristic structure refining process. First, the initial hidden nodes of the EBFNN model are selected randomly in the training samples, which are further partitioned into modules of hidden nodes with respect to their class labels. Subpopulations are initialized on modules, and the Co-CEA evolves all subpopulations to find the optimal EBFNN structural parameters. Then the heuristic structure refining process is performed on the individual in the elite pool with the special designed constructing and pruning operators. Finally, the CC-EBFNN model is tested on six real-world classification problems from the UCI machine learning repository, and experimental results illustrate that the EBFNN model can be estimated in fewer evolutionary trials, and is able to produce higher prediction accuracies with much simpler network structures when compared with conventional learning algorithms.

[1]  Qiuming Zhu,et al.  A global learning algorithm for a RBF network , 1999, Neural Networks.

[2]  Vladimir Pavlovic,et al.  Efficient discriminative learning of Bayesian network classifier , 2005 .

[3]  Horst Bischof,et al.  An efficient MDL-based construction of RBF networks , 1998, Neural Networks.

[4]  Geoffrey Holmes,et al.  A diagnostic tool for tree based supervised classification learning algorithms , 1999, ICONIP'99. ANZIIS'99 & ANNES'99 & ACNN'99. 6th International Conference on Neural Information Processing. Proceedings (Cat. No.99EX378).

[5]  Kenneth A. De Jong,et al.  The Coevolution of Antibodies for Concept Learning , 1998, PPSN.

[6]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[7]  Geoffrey I. Webb,et al.  MultiBoosting: A Technique for Combining Boosting and Wagging , 2000, Machine Learning.

[8]  Armando Blanco,et al.  A real-coded genetic algorithm for training recurrent neural networks , 2001, Neural Networks.

[9]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[10]  Haralambos Sarimveis,et al.  A new algorithm for developing dynamic radial basis function neural network models based on genetic algorithms , 2004, Comput. Chem. Eng..

[11]  Bernhard Pfahringer,et al.  Improving on Bagging with Input Smearing , 2006, PAKDD.

[12]  Peter W. Eklund.  Comparative study of public-domain supervised machine-learning accuracy on the UCI database , 1999, Defense, Security, and Sensing.

[13]  David Zhang,et al.  A fast kernel-based nonlinear discriminant analysis for multi-class problems , 2006, Pattern Recognit..

[14]  M. Mak,et al.  Estimation of Elliptical Basis Function Parameters by the Em Algorithm with Application to Speaker Veriication (final Version) Paper No.: Tnna069 , 2000 .

[15]  Peter W. Eklund,et al.  A Comparative Study of Public Domain Supervised Classifier Performance on the UCI Database , 2006, Aust. J. Intell. Inf. Process. Syst..

[16]  Luo Jiancheng,et al.  An Elliptical Basis Function Network for Classification of Remote-Sensing Images , 2005 .

[17]  Gao Daqi,et al.  Adaptive RBF neural networks for pattern classifications , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[18]  Chandrika Kamath,et al.  Inducing oblique decision trees with evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[19]  Dirk Thierens,et al.  The balance between proximity and diversity in multiobjective evolutionary algorithms , 2003, IEEE Trans. Evol. Comput..

[20]  Sorin Draghici,et al.  The constraint based decomposition (CBD) training architecture , 2001, Neural Networks.

[21]  Thomas G. Dietterich.  An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization , 2000, Machine Learning.

[22]  Nicolás García-Pedrajas,et al.  Immune Network based Ensembles , 2007, ESANN.

[23]  M. Mimura,et al.  A nonlinear equalizer based on estimation of RBF's centers , 1999, ISCAS'99. Proceedings of the 1999 IEEE International Symposium on Circuits and Systems VLSI (Cat. No.99CH36349).

[24]  Lipo Wang,et al.  Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[25]  Hyun-Chul Kim,et al.  Bayesian Gaussian Process Classification with the EM-EP Algorithm , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Tatsuo Higuchi,et al.  Evolutionary learning of nearest-neighbor MLP , 1996, IEEE Trans. Neural Networks.

[27]  Sotiris B. Kotsiantis,et al.  Logitboost of Simple Bayesian Classifier , 2005, Informatica.

[28]  Xiaodong Li,et al.  Parameter Control within a Co-operative Co-evolutionary Genetic Algorithm , 2002, PPSN.

[29]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[30]  R. Eriksson,et al.  Cooperative Coevolution in Inventory Control Optimisation , 1997, ICANNGA.

[31]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[32]  Chen Fu-zan.  GA-RBFNN learning algorithm for complex classifications , 2006 .

[33]  César Hervás-Martínez,et al.  Cooperative coevolution of artificial neural network ensembles for pattern classification , 2005, IEEE Transactions on Evolutionary Computation.

[34]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.