Evolving Feed-forward Neural Networks Through Evolutionary Mutation Parameters

In this paper we show a preliminary work on evolutionary mutation parameters in order to understand whether it is possible or not to skip mutation parameters tuning. In particular, rather than considering mutation parameters as global environmental features, we regard them as endogenous features of the individuals by putting them directly in the genotype. In this way we let the optimal values emerge from the evolutionary process itself. As case study, we apply the proposed methodology to the training of feed-forward neural netwoks on nine classification benchmarks and compare it to other five well established techniques. Results show the effectiveness of the proposed appraoch to get very promising results passing over the boring task of off-line optimal parameters tuning.

[1]  Matteo Matteucci,et al.  ELeaRNT: Evolutionary Learning of Rich Neural Network Topologies , 2006 .

[2]  David White,et al.  GANNet: A Genetic Algorithm for Optimizing Topology and Weights in Neural Network Design , 1993, IWANN.

[3]  H. Altay Güvenir,et al.  Classification by Voting Feature Intervals , 1997, ECML.

[4]  I. Bertini,et al.  Adaptivity of Artificial Life Environment for On-Line Optimization of Evolving Dynamical Systems , 2001 .

[5]  P Bourgine,et al.  Towards a Practice of Autonomous Systems , 1992 .

[6]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[7]  Claus Emmeche,et al.  The garden in the machine: the emerging science of artificial life , 1994 .

[8]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[9]  E. Cantu-Paz,et al.  An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[10]  Teresa B. Ludermir,et al.  Evolutionary Design Of Neural Networks: Application To River Flow Prediction , 2001 .

[11]  Jaehong Park,et al.  Evolutionary projection neural networks , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[12]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[13]  Geoffrey E. Hinton,et al.  Learning representations by back-propagation errors, nature , 1986 .

[14]  M. Bedau,et al.  Adaptation of Mutation Rates in a Simple Model of Evolution , 1994 .

[15]  Chandrika Kamath,et al.  Evolving Neural Networks for the Classification of Galaxies , 2002, GECCO.

[16]  Panos A. Ligomenides,et al.  GANNET: a genetic algorithm for searching topology and weight spaces in neural network design. The first step in finding a neural network solution , 1993 .

[17]  Claus Emmeche,et al.  The Garden in the Machine , 1994 .

[18]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[19]  David B. Fogel,et al.  Meta-evolutionary programming , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[20]  I. Bertini,et al.  EVOLUTIONARY FEED-FORWARD NEURAL NETWORKS FOR TRAFFIC PREDICTION , 2003 .

[21]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[22]  John G. Cleary,et al.  K*: An Instance-based Learner Using and Entropic Distance Measure , 1995, ICML.

[23]  Vasant Honavar,et al.  Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature , 1995 .

[24]  David Metzgar,et al.  Evidence for the Adaptive Evolution of Mutation Rates , 2000, Cell.

[25]  Mauro Annunziato,et al.  Artificial life and online flows optimisation in energy networks , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[26]  Jan T. Kim.  Energy dependent adaptation of mutation rates in computer models of evolution , 1998 .

[27]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[28]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[29]  Xin Yao,et al.  Evolutionary design of artificial neural networks with different nodes , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[30]  Geoffrey I. Webb,et al.  MultiBoosting: A Technique for Combining Boosting and Wagging , 2000, Machine Learning.