Design of artificial neural networks using a modified Particle Swarm Optimization algorithm

In the last years, bio-inspired algorithms have shown their power in different non-linear optimization problems. Due to the efficiency and adaptability of bio-inspired algorithms, in this paper we explore a new way to design an artificial neural network (ANN). For this task, a modified PSO algorithm was used. We do not only study the problem of finding the optimal synaptic weights of an ANN but also its topology and transfer functions. In other words, given a set of inputs and desired patterns, with the proposal we are able to find the best topology, the number of neurons, the transfer function for each neuron, as well as the synaptic weights. This allows to designing an ANN to be used to solve a given problem. The proposal is tested using several non-linear problems.

[1]  Kiyotaka Izumi,et al.  A particle-swarm-optimized fuzzy-neural network for voice-controlled robot systems , 2005, IEEE Transactions on Industrial Electronics.

[2]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[3]  Thomas Kiel Rasmussen,et al.  Hybrid Particle Swarm Optimiser with breeding and subpopulations , 2001 .

[4]  Juan Julián Merelo Guervós,et al.  Evolving Multilayer Perceptrons , 2000, Neural Processing Letters.

[5]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[6]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[7]  J. Kennedy,et al.  Stereotyping: improving particle swarm performance with cluster analysis , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[8]  Hitoshi Iba,et al.  Particle swarm optimization with Gaussian mutation , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[9]  Ge Xiurun,et al.  An improved PSO-based ANN with simulated annealing technique , 2005, Neurocomputing.

[10]  Patrick van der Smagt,et al.  Introduction to neural networks , 1995, The Lancet.

[11]  Cheng-Yan Kao,et al.  A Robust Evolutionary Algorithm for Training Neural Networks , 2001, Neural Computing & Applications.

[12]  Zhiwei Wang,et al.  Particle swarm optimization and neural network application for QSAR , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[13]  Kwok-wing Chau,et al.  Application of a PSO-based neural network in analysis of outcomes of construction claims , 2007 .

[14]  Daniel Rivero,et al.  Automatic Design of ANNs by Means of GP for Data Mining Tasks: Iris Flower Classification Problem , 2007, ICANNGA.

[15]  Christian Posthoff,et al.  Earthquake classifying neural networks trained with random dynamic neighborhood PSOs , 2007, GECCO '07.

[16]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[17]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[18]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[19]  James A. Anderson,et al.  An Introduction To Neural Networks , 1998 .

[20]  Liang Zhao,et al.  PSO-based single multiplicative neuron model for time series prediction , 2009, Expert Syst. Appl..

[21]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[22]  Mingquan Chen,et al.  Second Generation Particle Swarm Optimization , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[23]  Lifeng Xi,et al.  An Improved Particle Swarm Optimization for Evolving Feedforward Artificial Neural Networks , 2007, Neural Processing Letters.