Neighborhood centroid opposite-based learning Harris Hawks optimization for training neural networks

The Harris Hawks Optimization Algorithm is a new metaheuristic optimization that simulates the process of Harris Hawk hunting prey (rabbit) in nature. The global and local search processes of the algorithm are performed by simulating several stages of cooperative behavior during hunting. To enhance the performance of this algorithm, in this paper we propose a neighborhood centroid opposite-based learning Harris Hawks optimization algorithm (NCOHHO). The mechanism of applying the neighborhood centroid under the premise of using opposite-based learning technology to improve the performance of the algorithm, the neighborhood centroid is used as a reference point for the generation of the opposite particle, while maintaining the diversity of the population and make full use of the swarm search experience to expand the search range of the reverse solution. Enhancing the probability of finding the optimal solution and the improved algorithm is superior to the original Harris Hawks Optimization algorithm in all aspects. We apply NCOHHO to the training of feed-forward neural network (FNN). To confirm that using NCOHHO to train FNN is more effective, five classification datasets are applied to benchmark the performance of the proposed method. Comprehensive comparison and analysis from the three aspects of mean, variance and classification success rate, the experimental results show that the proposed NCOHHO algorithm for optimization FNN has the best comprehensive performance and has more outstanding performance than other metaheuristic algorithms in terms of the performance measures.

[1]  Rui Wang,et al.  Elite opposition-based flower pollination algorithm , 2016, Neurocomputing.

[2]  R. Palmer,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[3]  Ya-Qing Bi,et al.  New Proofs of Some q-Summation and q-Transformation Formulas , 2014, TheScientificWorldJournal.

[4]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[5]  Shingo Mabu,et al.  Training of Multi-Branch Neural Networks using RasID-GA , 2007, 2007 IEEE Congress on Evolutionary Computation.

[6]  Harun Uğuz,et al.  A novel particle swarm optimization algorithm with Levy flight , 2014, Appl. Soft Comput..

[7]  Ali Safa Sadiq,et al.  Magnetic Optimization Algorithm for training Multi Layer Perceptron , 2011, 2011 IEEE 3rd International Conference on Communication Software and Networks.

[8]  Andrew Lewis,et al.  Adaptive gbest-guided gravitational search algorithm , 2014, Neural Computing and Applications.

[9]  Hassan B. Kazemian,et al.  Identification of probe request attacks in WLANs using neural networks , 2013, Neural Computing and Applications.

[10]  M. A. Behrang,et al.  A Hybrid Neural Network and Gravitational Search Algorithm (HNNGSA) Method to Solve well known Wessinger's Equation , 2011 .

[11]  Sebastian Madgwick,et al.  Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011, 2011 IEEE International Conference on Rehabilitation Robotics.

[12]  Ya Li,et al.  A Novel Artificial Bee Colony Algorithm Based on Internal-Feedback Strategy for Image Template Matching , 2014, TheScientificWorldJournal.

[13]  Karin Strauss,et al.  Accelerating Deep Convolutional Neural Networks Using Specialized Hardware , 2015 .

[14]  Zhi Yuan,et al.  A new technique for optimal estimation of the circuit-based PEMFCs using developed Sunflower Optimization Algorithm , 2020 .

[15]  Christian Blum,et al.  An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training , 2007, Neural Computing and Applications.

[16]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[17]  Dervis Karaboga,et al.  Hybrid Artificial Bee Colony algorithm for neural network training , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[18]  W. Krzanowski,et al.  A Criterion for Determining the Number of Groups in a Data Set Using Sum-of-Squares Clustering , 1988 .

[19]  Siti Zaiton Mohd Hashim,et al.  Training feedforward neural networks using hybrid particle swarm optimization and gravitational search algorithm , 2012, Appl. Math. Comput..

[20]  Shahryar Rahnamayan,et al.  Computing opposition by involving entire population , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[21]  Bai Li,et al.  An evolutionary approach for image retrieval based on lateral inhibition , 2016 .

[22]  Anand Nayyar,et al.  Advances in Swarm Intelligence and Machine Learning for Optimizing Problems in Image Processing and Data Analytics (Part 1) , 2019 .

[23]  Rich Caruana,et al.  An empirical comparison of supervised learning algorithms , 2006, ICML.

[24]  Anne-Johan Annema Feed-forward neural networks , 1995 .

[25]  Jianzhong Xu,et al.  Hybrid Nelder–Mead Algorithm and Dragonfly Algorithm for Function Optimization and the Training of a Multilayer Perceptron , 2019 .

[26]  Robert J. Marks,et al.  Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks , 1999 .

[27]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[28]  K. Jermsittiparsert,et al.  New optimal design for a hybrid solar chimney, solid oxide electrolysis and fuel cell based on improved deer hunting optimization algorithm , 2020 .

[29]  Hamid R. Tizhoosh,et al.  Opposition-Based Learning: A New Scheme for Machine Intelligence , 2005, International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06).

[30]  DeLiang Wang,et al.  Unsupervised Learning: Foundations of Neural Computation , 2001, AI Mag..

[31]  Navid Razmjooy,et al.  A New Meta-Heuristic Optimization Algorithm Inspired by FIFA World Cup Competitions: Theory and Its Application in PID Designing for AVR System , 2016 .

[32]  Anand Nayyar,et al.  Advances in Swarm Intelligence for Optimizing Problems in Computer Science , 2018 .

[33]  Seyed Mohammad Mirjalili,et al.  Designing evolutionary feedforward neural networks using social spider optimization algorithm , 2015, Neural Computing and Applications.

[34]  William Nick Street,et al.  Breast Cancer Diagnosis and Prognosis Via Linear Programming , 1995, Oper. Res..

[35]  Mohammad Ali Ghorbani,et al.  Design and implementation of a hybrid MLP-GSA model with multi-layer perceptron-gravitational search algorithm for monthly lake water level forecasting , 2018, Stochastic Environmental Research and Risk Assessment.

[36]  Dervis Karaboga,et al.  Artificial Bee Colony (ABC) Optimization Algorithm for Training Feed-Forward Neural Networks , 2007, MDAI.

[37]  Anand Nayyar,et al.  Introduction to Swarm Intelligence , 2018, Advances in Swarm Intelligence for Optimizing Problems in Computer Science.

[38]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[39]  Amir Hossein Gandomi,et al.  Opposition-based krill herd algorithm with Cauchy mutation and position clamping , 2016, Neurocomputing.

[40]  Kang Li,et al.  GA BASED NEURAL NETWORK MODELING OF NOX EMISSION IN A COAL-FIRED POWER GENERATION PLANT , 2002 .

[41]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[42]  P. Bhartia,et al.  Standard ozone profiles from balloon and satellite data sets , 1983 .

[43]  Silke A.T. Weber,et al.  Social-Spider Optimization-Based Artificial Neural Networks Training and Its Applications for Parkinson's Disease Identification , 2014, 2014 IEEE 27th International Symposium on Computer-Based Medical Systems.

[44]  Raymond Y. K. Lau,et al.  Least Squares Generative Adversarial Networks , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Zong Woo Geem,et al.  A New Heuristic Optimization Algorithm: Harmony Search , 2001, Simul..

[46]  Dinesh Gopalani,et al.  Numerical optimization and feed-forward neural networks training using an improved optimization algorithm: multiple leader salp swarm algorithm , 2019, Evolutionary Intelligence.

[47]  Jan Pieter Abrahams,et al.  Structure at 2.8 Â resolution of F1-ATPase from bovine heart mitochondria , 1994, Nature.

[48]  David Mazières,et al.  Kademlia: A Peer-to-Peer Information System Based on the XOR Metric , 2002, IPTPS.

[49]  Seyed Mohammad Mirjalili How effective is the Grey Wolf optimizer in training multi-layer perceptrons , 2014, Applied Intelligence.

[50]  R. Prentice,et al.  Regression analysis of grouped survival data with application to breast cancer data. , 1978, Biometrics.

[51]  Shahryar Rahnamayan,et al.  Opposition-Based Differential Evolution Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[52]  Christian Blum,et al.  Training feed-forward neural networks with ant colony optimization: an application to pattern classification , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[53]  M. Georgiopoulos,et al.  Feed-forward neural networks , 1994, IEEE Potentials.

[54]  Hao Ni,et al.  Recurrent Neural Network Based Language Model Adaptation for Accent Mandarin Speech , 2016, CCPR.

[55]  Pandian Vasant,et al.  Random Matrix Generators for Optimizing a Fuzzy Biofuel Supply Chain System , 2020, J. Adv. Eng. Comput..

[56]  Hossam Faris,et al.  Harris hawks optimization: Algorithm and applications , 2019, Future Gener. Comput. Syst..

[57]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[58]  Xin-She Yang,et al.  Engineering Optimization: An Introduction with Metaheuristic Applications , 2010 .

[59]  Harrison,et al.  [IEEE 2011 IEEE 12th International Conference on Rehabilitation Robotics: Reaching Users & the Community (ICORR 2011) - Zurich (2011.06.29-2011.07.1)] 2011 IEEE International Conference on Rehabilitation Robotics - Estimation of IMU and MARG orientation using a gradient descent algorithm , 2011 .

[60]  Yongli Wang,et al.  Selfish herds optimization algorithm with orthogonal design and information update for training multi-layer perceptron neural network , 2018, Applied Intelligence.

[61]  Yan-Peng Liu,et al.  Evolving Neural Networks Using the Hybrid of Ant Colony Optimization and BP Algorithms , 2006, ISNN.

[62]  Tara N. Sainath,et al.  Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[63]  M.M.A. Salama,et al.  Opposition-Based Differential Evolution , 2008, IEEE Transactions on Evolutionary Computation.

[64]  Chris Bishop,et al.  Improving the Generalization Properties of Radial Basis Function Neural Networks , 1991, Neural Computation.

[65]  Le Zhang,et al.  Developing a Novel Hybrid Biogeography-Based Optimization Algorithm for Multilayer Perceptron Training under Big Data Challenge , 2018, Sci. Program..