Design of structural modular neural networks with genetic algorithm

The back-propagation (BP) neural network and the radial basis function (RBF) neural network have been widely used in many engineering applications. In general, the BP neural network can better construct the global approximations to the input-output mapping, whereas an RBF neural network employs the exponentially decaying localized input-output mapping which can effectively model the large variation locally. In this paper, a structural modular neural network, by combining the BP neurons and the RBF neurons at the hidden layer, is proposed to construct a better input-output mapping both locally and globally. The use of genetic algorithm in searching the best hidden neurons makes the structural modular neural network less likely to be trapped in local minima than the traditional gradient-based search algorithms. An analytical function-the peak function is used first to assess the accuracy of the proposed modular approach. The verified approach is then applied to the strength model of concrete under triaxial stresses. The preliminary results show that the modular approach is more accurate than using the RBF neural network or the BP neural network alone.

[1]  Martin P. DeSimio,et al.  MLP iterative construction algorithm , 1997, Neurocomputing.

[2]  Mohamed S. Kamel,et al.  Modular neural networks: a survey. , 1999, International journal of neural systems.

[3]  Hean-Lee Poh,et al.  Analysis of Pruning in Backpropagation Networks for Artificial and Real Worls Mapping Problems , 1995, IWANN.

[4]  Michael Georgiopoulos,et al.  Coupling weight elimination with genetic algorithms to reduce network size and preserve generalization , 1997, Neurocomputing.

[5]  Steven K. Rogers,et al.  An Introduction to Biological and Artificial Neural Networks for Pattern Recognition , 1991 .

[6]  L.F.A. Wessels,et al.  Extrapolation and interpolation in neural network classifiers , 1992, IEEE Control Systems.

[7]  Michael C. Mozer,et al.  Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment , 1988, NIPS.

[8]  Sönke Johannes,et al.  Hierarchical visual stimuli: electrophysiological evidence for separate left hemispheric global and local processing mechanisms in humans , 1996, Neuroscience Letters.

[9]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[10]  Stephen A. Billings,et al.  Radial basis function network configuration using genetic algorithms , 1995, Neural Networks.

[11]  S. Shipp,et al.  The functional logic of cortical connections , 1988, Nature.

[12]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1992, Math. Control. Signals Syst..

[13]  Zhiye Zhao,et al.  An optimal neural network and concrete strength modeling , 2002 .

[14]  Jie Zhang,et al.  A Sequential Learning Approach for Single Hidden Layer Neural Networks , 1998, Neural Networks.

[15]  Rajesh Parekh,et al.  Constructive Neural Network Learning Algorithms for Multi-Category Real-Valued Pattern Classification , 1997 .

[16]  Michael C. Mozer,et al.  Using Relevance to Reduce Network Size Automatically , 1989 .

[17]  Yoshio Hirose,et al.  Backpropagation algorithm which varies the number of hidden units , 1989, International 1989 Joint Conference on Neural Networks.

[18]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[19]  Tomas Hrycej.  Modular learning in neural networks - a modularized approach to neural network classification , 1992, Sixth-Generation computer technology series.

[20]  Michael D. Kotsovos,et al.  A mathematical description of the strength properties of concrete under generalized stress , 1979 .

[21]  Panos A. Ligomenides,et al.  GANNET: a genetic algorithm for searching topology and weight spaces in neural network design. The first step in finding a neural network solution , 1993 .

[22]  Vasant Honavar,et al.  Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature , 1995 .

[23]  Ernesto Tarantino,et al.  Optimizing Neural Networks for Time Series Prediction , 1999 .

[24]  Somnath Mukhopadhyay,et al.  A polynomial time algorithm for the construction and training of a class of multilayer perceptrons , 1993, Neural Networks.

[25]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[26]  Anna Maria Fanelli,et al.  A Method of Pruning Layered Feed-Forward Neural Networks , 1993, IWANN.