A NOVEL CHEMICAL REACTION OPTIMIZATION ALGORITHM FOR HIGHER ORDER NEURAL NETWORK TRAINING

In this paper, an application of a novel chemical reaction optimization (CRO) algorithm for training higher order neural networks (HONNs), especially the Pi-Sigma Network (PSN) has been presented. In contrast to basic CRO algorithms, the proposed CRO algorithm used to train HONN possesses two modifications. The reactant size (population size) remains fixed throughout all the iteration, which makes it easier to implement; and adaptive chemical reactions followed by a strictly greedy reversible reaction have been used which assist to reach the global minima in less number of iterations. The performance of proposed algorithm for HONN training is evaluated through a well-known neural network training benchmark i.e. to classify the parity-p problems. The results obtained from the proposed algorithm to train HONN have been compared with results from the following algorithms: basic CRO algorithm and the two most popular variants of differential evolution algorithm (DE/rand/1/bin and DE/best/1/bin). It is observed that the application of the proposed CRO algorithm to HONN training (CRO-HONNT) performs statistically better than that of other algorithms.

[1]  Bilal Alatas,et al.  ACROA: Artificial Chemical Reaction Optimization Algorithm for global optimization , 2011, Expert Syst. Appl..

[2]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[3]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[4]  Michael G. Epitropakis,et al.  Hardware-friendly Higher-Order Neural Network Training using Distributed Evolutionary Algorithms , 2010, Appl. Soft Comput..

[5]  M.M.A. Salama,et al.  Opposition-Based Differential Evolution , 2008, IEEE Transactions on Evolutionary Computation.

[6]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[7]  De-Shuang Huang,et al.  Zeroing polynomials using modified constrained neural network approach , 2005, IEEE Transactions on Neural Networks.

[8]  Stavros J. Perantonis,et al.  Constrained Learning in Neural Networks: Application to Stable Factorization of 2-D Polynomials , 2004, Neural Processing Letters.

[9]  Victor O. K. Li,et al.  Real-Coded Chemical Reaction Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[10]  Victor O. K. Li,et al.  Chemical Reaction Optimization: a tutorial , 2012, Memetic Computing.

[11]  D. Pham,et al.  THE BEES ALGORITHM, A NOVEL TOOL FOR COMPLEX OPTIMISATION PROBLEMS , 2006 .

[12]  C. Moorehead.  All rights reserved , 1997 .

[13]  Joydeep Ghosh,et al.  Realization of Boolean Functions Using Binary Pi-sigma Networks , 1991 .

[14]  Joydeep Ghosh,et al.  The pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[15]  Lin Zhu,et al.  Ant colony optimization for continuous domains , 2012, 2012 8th International Conference on Natural Computation.

[16]  A. Diop.  Journal of Theoretical and Applied Information Technology , 2012 .

[17]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[18]  Bilal Alatas,et al.  A novel chemistry based metaheuristic optimization method for mining of classification rules , 2012, Expert Syst. Appl..

[19]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[20]  Victor O. K. Li,et al.  Chemical-Reaction-Inspired Metaheuristic for Optimization , 2010, IEEE Transactions on Evolutionary Computation.

[21]  Marco Dorigo,et al.  Ant colony optimization for continuous domains , 2008, Eur. J. Oper. Res..

[22]  Jong-Hwan Kim,et al.  Quantum-inspired evolutionary algorithm for a class of combinatorial optimization , 2002, IEEE Trans. Evol. Comput..

[23]  Tung Khac Truong,et al.  Chemical reaction optimization with greedy strategy for the 0-1 knapsack problem , 2013, Appl. Soft Comput..

[24]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[25]  Joydeep Ghosh,et al.  Efficient Higher-Order Neural Networks for Classification and Function Approximation , 1992, Int. J. Neural Syst..

[26]  Victor O. K. Li,et al.  Evolutionary artificial neural network based on Chemical Reaction Optimization , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).