Adaptive Structural Optimisation of Neural Networks

Structural design of an artificial neural network (ANN) is a very important phase in the construction of such a network. The selection of the optimal number of hidden layers and hidden nodes has a significant impact on the performance of a neural network, though typically decided in an adhoc manner. In this paper, the structure of a neural network is adaptively optimised by determine the number of hidden layers and hidden nodes that give the optimal performance in a given problem domain. Two optimisation approaches have been developed based on the Particle Swarm Optimisation (PSO) algorithm, which is an evolutionary algorithm which uses a cooperative approach. These approaches have been applied on two well known case studies in the classification domain, namely the Iris data classification and the Ionosphere data classification. The obtained results and comparisons done with past research work has clearly shown that this method of optimisation is by far, the best approach for adaptive structural optimisation of ANNs. Keywords: neural networks, particle swarm optimization, weight adjestment, hidden layer adjustment. doi: 10.4038/icter.v1i1.450 The International Journal on Advances in ICT for Emerging Regions 2008 01 (01) : 33 - 41

[1]  Russell C. Eberhart,et al.  Human tremor analysis using particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[2]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[3]  Andries Petrus Engelbrecht,et al.  Cooperative learning in neural networks using particle swarm optimizers , 2000, South Afr. Comput. J..

[4]  Ben Niu,et al.  A Hybrid Particle Swarm Optimization for Feed-Forward Neural Network Training , 2008, ICIC.

[5]  Dexian Huang,et al.  Designing Neural Networks Using Hybrid Particle Swarm Optimization , 2005, ISNN.

[6]  Vasant Honavar,et al.  Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature , 1995 .

[7]  Yu Li,et al.  Particle swarm optimisation for evolving artificial neural network , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[8]  Andries P. Engelbrecht,et al.  Effects of swarm size on Cooperative Particle Swarm Optimisers , 2001 .

[9]  Martin Eldracher.  Classification of non-linear-separable real-world-problems using Δ-rule, perceptrons, and topologically distributed encoding , 1992, SAC '92.

[10]  James Blondin,et al.  Particle Swarm Optimization: A Tutorial , 2009 .

[11]  M. Teshnehlab,et al.  Training ANFIS structure with modified PSO algorithm , 2007, 2007 Mediterranean Conference on Control & Automation.