Hybrid Evolutionary Algorithm with Product-Unit Neural Networks for Classification

In this paper we propose a classification method based on a special class of feed-forward neural network, namely product-unit neural networks, and on a dynamic version of a hybrid evolutionary neural network algorithm. The method combines an evolutionary algorithm, a clustering process, and a local search procedure, where the clustering process and the local search are only applied at specific stages of the evolutionary process. Our results with the product-unit models and the evolutionary approach show a very interesting performance in terms of classification accuracy, yielding a state-of-the-art performance.

[1]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[2]  Andries Petrus Engelbrecht,et al.  Global optimization algorithms for training product unit neural networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[3]  James F. Frenzel,et al.  Training product unit neural networks with genetic algorithms , 1993, IEEE Expert.

[4]  Kazumi Saito,et al.  Extracting regression rules from neural networks , 2002, Neural Networks.

[5]  João Gama,et al.  Functional Trees , 2001, Machine Learning.

[6]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[7]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[8]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[9]  César Hervás-Martínez,et al.  Evolutionary product unit based neural networks for regression , 2006, Neural Networks.

[10]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[11]  David E. Rumelhart,et al.  Product Units: A Computationally Powerful and Biologically Plausible Extension to Backpropagation Networks , 1989, Neural Computation.

[12]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[13]  Michael Schmitt,et al.  On the Complexity of Computing and Learning with Multiplicative Neural Networks , 2002, Neural Computation.

[14]  Eibe Frank,et al.  Logistic Model Trees , 2003, ECML.