Artificial Neural Network Weight Optimization: A Review

Optimizing the weights of Artificial Neural Networks (ANNs) is a great important of a complex task in the research of machine learning due to dependence of its performance to the success of learning process and the training method. This paper reviews the implementation of meta-heuristic algorithms in ANNs’ weight optimization by studying their advantages and disadvantages giving consideration to some meta-heuristic members such as Genetic algorithim, Particle Swarm Optimization and recently introduced meta-heuristic algorithm called Harmony Search Algorithm (HSA). Also, the application of local search based algorithms to optimize the ANNs weights and their benefits as well as their limitations are briefly elaborated. Finally, a comparison between local search methods and global optimization methods is carried out to speculate the trends in the progresses of ANNs’ weight optimization in the current resrearch.

[1]  Xin-She Yang.  Harmony Search as a Metaheuristic Algorithm , 2009 .

[2]  Bingxue Shi,et al.  Hybrid BP-GA for multilayer feedforward neural networks , 2000, ICECS 2000. 7th IEEE International Conference on Electronics, Circuits and Systems (Cat. No.00EX445).

[3]  Fan Liu,et al.  Genetic Algorithms for MLP Neural Network parameters optimization , 2009, 2009 Chinese Control and Decision Conference.

[4]  Ali Kattan,et al.  TRAINING FEED-FORWARD ARTIFICIAL NEURAL NETWORKS FOR PATTERN-CLASSIFICATION USING THE HARMONY SEARCH ALGORITHM , 2013, DEIS 2013.

[5]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[6]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[7]  Minghui Jiang,et al.  Hybrid Neural Network Based on GA-BP for Personal Credit Scoring , 2008, 2008 Fourth International Conference on Natural Computation.

[8]  Masaya Yoshikawa,et al.  Ant Colony Optimization Routing Algorithm with Tabu Search , 2010 .

[9]  Hui Li,et al.  Evolutionary artificial neural networks: a review , 2011, Artificial Intelligence Review.

[10]  Liang Li,et al.  A Combinatorial Search Method Based on Harmony Search Algorithm and Particle Swarm Optimization in Slope Stability Analysis , 2009, 2009 International Conference on Computational Intelligence and Software Engineering.

[11]  John Paul,et al.  Tetris Agent Optimization Using Harmony Search Algorithm , 2011 .

[12]  Fernando José Von Zuben,et al.  Training multilayer perceptrons with a Gaussian Artificial Immune System , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[13]  Ali Kattan,et al.  Harmony Search Based Supervised Training of Artificial Neural Networks , 2010, 2010 International Conference on Intelligent Systems, Modelling and Simulation.

[14]  H. Shayeghi,et al.  A HYBRID PARTICLE SWARM OPTIMIZATION BACK PROPAGATION ALGORITHM FOR SHORT TERM LOAD FORECASTING , 2011 .

[15]  Ming Zhu,et al.  Intelligent trading using support vector regression and multilayer perceptrons optimized with genetic algorithms , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[16]  Siti Mariyam Shamsuddin,et al.  Particle Swarm Optimization For Neural Network Learning Enhancement , 2008 .

[17]  Andres M. Gonzalez,et al.  Evolutionary Algorithms for Selecting the Architecture of a MLP Neural Network: A Credit Scoring Case , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[18]  Mohammad Reza Razfar,et al.  Constrained optimum surface roughness prediction in turning of X20Cr13 by coupling novel modified harmony search-based neural network and modified harmony search algorithm , 2012 .

[19]  Alireza Rezazadeh,et al.  Artificial neural network training using a new efficient optimization algorithm , 2013, Appl. Soft Comput..

[20]  Saeed Tavakoli,et al.  Feedforward neural network training using intelligent global harmony search , 2012, Evolving Systems.

[21]  Teresa Bernarda Ludermir,et al.  Hybrid Training Method for MLP: Optimization of Architecture and Training , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[22]  Qiongxin Liu,et al.  A BP Neural Network Model Based on Genetic Algorithm for Comprehensive Evaluation , 2011, 2011 Third Pacific-Asia Conference on Circuits, Communications and System (PACCS).

[23]  Miguel Angel Guevara Lopez,et al.  Optimizing the Area Under the ROC Curve in Multilayer Perceptron-based Classifiers , 2011 .

[24]  Lale Özbakir,et al.  Training neural networks with harmony search algorithms for classification problems , 2012, Eng. Appl. Artif. Intell..

[25]  I. M. El-Henawy,et al.  Predicting stock index using neural network combined with evolutionary computation methods , 2010, 2010 The 7th International Conference on Informatics and Systems (INFOS).

[26]  Teresa B. Ludermir,et al.  Particle Swarm Optimization of Neural Network Architectures and Weights , 2007 .

[27]  Wei Gao,et al.  Evolutionary Neural Network Based on New Ant Colony Algorithm , 2008, 2008 International Symposium on Computational Intelligence and Design.

[28]  Jing J. Liang,et al.  A self-adaptive global best harmony search algorithm for continuous optimization problems , 2010, Appl. Math. Comput..

[29]  Alireza Sadeghian,et al.  A Variation of Particle Swarm Optimization for Training of Artificial Neural Networks , 2010 .