Structure optimization of neural network for dynamic system modeling using multi-objective genetic algorithm

The problem of constructing an adequate and parsimonious neural network topology for modeling non-linear dynamic system is studied and investigated. Neural networks have been shown to perform function approximation and represent dynamic systems. The network structures are usually guessed or selected in accordance with the designer’s prior knowledge. However, the multiplicity of the model parameters makes it troublesome to get an optimum structure. In this paper, an alternative algorithm based on a multi-objective optimization algorithm is proposed. The developed neural network model should fulfil two criteria or objectives namely good predictive accuracy and minimum model structure. The result shows that the proposed algorithm is able to identify simulated examples correctly, and identifies the adequate model for real process data based on a set of solutions called the Pareto optimal set, from which the best network can be selected.

[1]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[2]  Rich Caruana,et al.  Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping , 2000, NIPS.

[3]  Hussein A. Abbass,et al.  Speeding Up Backpropagation Using Multiobjective Evolutionary Algorithms , 2003, Neural Computation.

[4]  C. Fonseca,et al.  GENETIC ALGORITHMS FOR MULTI-OBJECTIVE OPTIMIZATION: FORMULATION, DISCUSSION, AND GENERALIZATION , 1993 .

[5]  Witold Pedrycz,et al.  Genetic optimization-driven multi-layer hybrid fuzzy neural networks , 2006, Simul. Model. Pract. Theory.

[6]  Martin T. Hagan,et al.  Neural network design , 1995 .

[7]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[8]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[9]  S Usui,et al.  Robustness, evolvability, and optimality of evolutionary neural networks. , 2005, Bio Systems.

[10]  Randall S. Sexton,et al.  Simultaneous optimization of neural network function and architecture algorithm , 2004, Decis. Support Syst..

[11]  Jonathan E. Fieldsend,et al.  Pareto evolutionary neural networks , 2005, IEEE Transactions on Neural Networks.

[12]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[13]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[14]  Sung-Kwun Oh,et al.  A genetic approach to modeling fuzzy systems based on information granulation and successive generation-based evolution method , 2007, Simul. Model. Pract. Theory.

[15]  S. Billings,et al.  Correlation based model validity tests for non-linear models , 1986 .

[16]  Yuanyuan Zhang,et al.  Improvements of nonlinear dynamic modeling of hot-film MAF sensor , 2008 .

[17]  M. Georgiopoulos,et al.  Feed-forward neural networks , 1994, IEEE Potentials.

[18]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[19]  R. K. Ursem.  Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[20]  R.J.F. Dow,et al.  Neural net pruning-why and how , 1988, IEEE 1988 International Conference on Neural Networks.

[21]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[22]  Surjya K. Pal,et al.  Modeling of electrical discharge machining process using back propagation neural network and multi-objective optimization using non-dominating sorting genetic algorithm-II , 2007 .

[23]  Mohd Azlan Hussain,et al.  Model structure selection for a discrete-time non-linear system using a genetic algorithm , 2004 .

[24]  Mohamed Ibnkahla,et al.  Nonlinear System Identification Using Neural Networks Trained with Natural Gradient Descent , 2003, EURASIP J. Adv. Signal Process..

[25]  Stephen A. Billings,et al.  Non-linear system identification using neural networks , 1990 .

[26]  J. D. Schaffer,et al.  Some experiments in machine learning using vector evaluated genetic algorithms (artificial intelligence, optimization, adaptation, pattern recognition) , 1984 .

[27]  Niels Kjølstad Poulsen,et al.  Neural Networks for Modelling and Control of Dynamic Systems: A Practitioner’s Handbook , 2000 .

[28]  Stephen A. Billings,et al.  Properties of neural networks with applications to modelling non-linear dynamical systems , 1992 .

[29]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[30]  Héctor Pomares,et al.  Multiobjective evolutionary optimization of the size, shape, and position parameters of radial basis function networks for function approximation , 2003, IEEE Trans. Neural Networks.

[31]  Randall S. Sexton,et al.  Optimization of neural networks: A comparative analysis of the genetic algorithm and simulated annealing , 1999, Eur. J. Oper. Res..