Percentage-Based Hybrid Pattern Training with Neural Network Specific Cross Over

In this paper, a new weight-setting method is proposed to improve the training time and generalization accuracy of feed-forward neural networks. This method introduces a percentage-based hybrid pattern training (PHP) scheme and aims to provide a solution to the problem dependency of other Genetic Algorithm (GA)-based Neural Network weight-setting methods. A neural network is trained using a neural network specific GA until a certain percentage of the training patterns is learned. The weights thus obtained are used as the initial weights for backpropagation (BP) training, which is then applied to complete the network training. Further improvement to the method was looked into and the use of a distributed GA in the weight-setting phase was investigated. The final approach derived was tested on four neural network problems—we observed that as the number of patterns trained using GA approaches 50% of the total number of training patterns, the proposed method is more effective in pulling the networks out of local minima. Additionally, the networks trained using this method showed as much as 75% improvement in training time and 15% improvement in generalization accuracy.

[1]  Darrell Whitley,et al.  Genetic cascade learning for neural networks , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[2]  Steven Guan,et al.  Incremental Learning with Respect to New Incoming Input Attributes , 2004, Neural Processing Letters.

[3]  Jong Beom Ra,et al.  Weight value initialization for improving training speed in the backpropagation network , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[4]  Hussein Alnuweiri,et al.  Acceleration of back propagation through initial weight pre-training with delta rule , 1993, IEEE International Conference on Neural Networks.

[5]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[6]  Alan F. Murray,et al.  IEEE International Conference on Neural Networks , 1997 .

[7]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[8]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[9]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[10]  Stefan Bornholdt,et al.  General asymmetric neural networks and structure design by genetic algorithms: a learning rule for temporal patterns , 1992, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[11]  John H. Holland,et al.  Genetic Algorithms and the Optimal Allocation of Trials , 1973, SIAM J. Comput..

[12]  David J. Montana,et al.  A Weighted Probabilistic Neural Network , 1991, NIPS.

[13]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[14]  Jan Torreele,et al.  Temporal Processing with Recurrent Networks: An Evolutionary Approach , 1991, ICGA.

[15]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[16]  David R. Jefferson,et al.  An Artificial Neural Network Representation for Artificial Organisms , 1990, PPSN.

[17]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[18]  Mitchell A. Potter,et al.  A genetic cascade-correlation learning algorithm , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[19]  Wenjun Zhuang,et al.  Surface roughness modelling with neural networks , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[20]  Rudolf Kruse,et al.  A fuzzy neural network learning fuzzy control rules and membership functions by fuzzy error backpropagation , 1993, IEEE International Conference on Neural Networks.

[21]  Moritoshi Yasunaga,et al.  Parallel back-propagation using genetic algorithm: real-time BP learning on the massively parallel computer CP-PACS , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[22]  Kangling Fang,et al.  A neural network controller based on genetic algorithms , 1997, 1997 IEEE International Conference on Intelligent Processing Systems (Cat. No.97TH8335).

[23]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .