Recursive percentage based hybrid pattern (RPHP) training for curve fitting

In this paper, we present the RPHP training algorithm, which finds several good local optimal points (pseudo global optima) automatically using an efficient combination of global and local search algorithms. This overcomes the problem of supervised learning algorithms being trapped in a local optima. Further, to solve a test pattern, we use a modified version of the Kth nearest neighbor (KNN) algorithm as a second level pattern distributor. We tested our approach on three curve fitting problems, whose coefficients were estimated both using genetic algorithms and the RPHP algorithm. The problems were chosen such that they had a small probability of finding a global optimal solution. It was found that the RPHP algorithms performed faster and improved generalization accuracy by as much as 25%.

[1]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[2]  T. Ash,et al.  Dynamic node creation in backpropagation networks , 1989, International 1989 Joint Conference on Neural Networks.

[3]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[4]  Hiroaki Kitano,et al.  A Hybrid Search for Genetic Algorithms: Combining Genetic AlgorithmsTABU Searchand Simulated Annealing , 1993, ICGA.

[5]  L. Darrell Whitley,et al.  Genetic algorithms and neural networks: optimizing connections and connectivity , 1990, Parallel Comput..

[6]  Tariq Samad,et al.  Towards the Genetic Synthesisof Neural Networks , 1989, ICGA.

[7]  David J. Montana,et al.  A Weighted Probabilistic Neural Network , 1991, NIPS.

[8]  Hugues Bersini,et al.  The immune and the chemical crossover , 2002, IEEE Trans. Evol. Comput..

[9]  Jan Torreele,et al.  Temporal Processing with Recurrent Networks: An Evolutionary Approach , 1991, ICGA.

[10]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[11]  Stefan Bornholdt,et al.  General asymmetric neural networks and structure design by genetic algorithms: a learning rule for temporal patterns , 1992, Proceedings of IEEE Systems Man and Cybernetics Conference - SMC.

[12]  Hugues Bersini,et al.  In Search of a Good Evolution-Optimization Crossover , 1992, PPSN.

[13]  David E. Goldberg,et al.  A Genetic Algorithm for Parallel Simulated Annealing , 1992, PPSN.

[14]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[15]  Hajime Kita,et al.  A parallel and modular multi-sieving neural network architecture for constructive learning , 1995 .

[16]  Darrell Whitley,et al.  Genetic cascade learning for neural networks , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[17]  John R. Koza,et al.  Genetic generation of both the weights and architecture for a neural network , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[18]  Mitchell A. Potter,et al.  A genetic cascade-correlation learning algorithm , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[19]  T. Yu,et al.  GENETIC PROGRAMMING : THEORY AND PRACTICE , 2005 .

[20]  R. R. Saldanha,et al.  Improvements in genetic algorithms , 2001 .

[21]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[22]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[23]  Moritoshi Yasunaga,et al.  Parallel back-propagation using genetic algorithm: real-time BP learning on the massively parallel computer CP-PACS , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[24]  Chunyu Bao,et al.  Task Decomposition Using Pattern Distributor , 2004 .

[25]  Jongwan Kim,et al.  Multiple neural networks using the reduced input dimension , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[26]  David R. Jefferson,et al.  An Artificial Neural Network Representation for Artificial Organisms , 1990, PPSN.

[27]  John H. Holland,et al.  Genetic Algorithms and the Optimal Allocation of Trials , 1973, SIAM J. Comput..

[28]  Hsin-Chia Fu,et al.  A divide-and-conquer methodology for modular supervised neural network design , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).