A novel hybrid sine cosine algorithm for global optimization and its application to train multilayer perceptrons

The Sine Cosine Algorithm (SCA) is a recently developed efficient metaheuristic algorithm to find the solution of global optimization problems. However, in some circumstances, this algorithm suffers the problem of low exploitation, skipping of true solutions and insufficient balance between exploration and exploitation. Therefore, the present paper aims to alleviate these issues from SCA by proposing an improved variant of SCA called HSCA. The HSCA modifies the search mechanism of classical SCA by including the leading guidance and hybridizing with simulated quenching algorithm. The proposed HSCA is tested on classical benchmark set, standard and complex benchmarks sets IEEE CEC 2014 and CEC 2017 and four engineering optimization problems. In addition to these problems, the HSCA is also used to train multilayer perceptrons as a real-life application. The experimental results and analysis on benchmark problems and real-life application problems demonstrate the superiority of the HSCA as compared to other comparative optimization algorithms.

[1]  Shuihua Wang,et al.  Combining extreme learning machine with modified sine cosine algorithm for detection of pathological brain , 2018, Comput. Electr. Eng..

[2]  Carlos A. Coello Coello,et al.  An empirical study about the usefulness of evolution strategies to solve constrained optimization problems , 2008, Int. J. Gen. Syst..

[3]  T.,et al.  Training Feedforward Networks with the Marquardt Algorithm , 2004 .

[4]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[5]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[6]  Ardeshir Bahreininejad,et al.  Mine blast algorithm: A new population based algorithm for solving constrained engineering optimization problems , 2013, Appl. Soft Comput..

[7]  Aboul Ella Hassanien,et al.  Training feedforward neural networks using Sine-Cosine algorithm to improve the prediction of liver enzymes on fish farmed on nano-selenite , 2016, 2016 12th International Computer Engineering Conference (ICENCO).

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Ling Wang,et al.  An effective co-evolutionary differential evolution for constrained optimization , 2007, Appl. Math. Comput..

[10]  Peter Rossmanith,et al.  Simulated Annealing , 2008, Taschenbuch der Algorithmen.

[11]  Aboul Ella Hassanien,et al.  Sine cosine optimization algorithm for feature selection , 2016, 2016 International Symposium on INnovations in Intelligent SysTems and Applications (INISTA).

[12]  Hossein Nezamabadi-pour,et al.  GSA: A Gravitational Search Algorithm , 2009, Inf. Sci..

[13]  B. Kappen.  Minimizing the System Error in Feedforward Neural Networks with Evolution Strategy , 2022 .

[14]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[15]  M. Hariharan,et al.  Sine–cosine algorithm for feature selection with elitism strategy and new updating mechanism , 2017, Neural Comput. Appl..

[16]  Jasbir S. Arora,et al.  4 – Optimum Design Concepts , 2004 .

[17]  Zhongliang Deng,et al.  An improved sine cosine algorithm based on levy flight , 2017, International Conference on Digital Image Processing.

[18]  S. N. Kramer,et al.  An Augmented Lagrange Multiplier Based Method for Mixed Integer Discrete Continuous Optimization and Its Applications to Mechanical Design , 1994 .

[19]  Seyed Mohammad Mirjalili,et al.  The Ant Lion Optimizer , 2015, Adv. Eng. Softw..

[20]  Seyed Mohammad Mirjalili,et al.  Moth-flame optimization algorithm: A novel nature-inspired heuristic paradigm , 2015, Knowl. Based Syst..

[21]  S. Wu,et al.  GENETIC ALGORITHMS FOR NONLINEAR MIXED DISCRETE-INTEGER OPTIMIZATION PROBLEMS VIA META-GENETIC PARAMETER OPTIMIZATION , 1995 .

[22]  Mostafa Meshkat,et al.  A novel weighted update position mechanism to improve the performance of sine cosine algorithm , 2017, 2017 5th Iranian Joint Congress on Fuzzy and Intelligent Systems (CFIS).

[23]  Diego Oliva,et al.  An improved Opposition-Based Sine Cosine Algorithm for global optimization , 2017, Expert Syst. Appl..

[24]  Seyedali Mirjalili,et al.  SCA: A Sine Cosine Algorithm for solving optimization problems , 2016, Knowl. Based Syst..

[25]  Chenglin Wen,et al.  Deep learning fault diagnosis method based on global optimization GAN for unbalanced data , 2020, Knowl. Based Syst..

[26]  Andrew Lewis,et al.  The Whale Optimization Algorithm , 2016, Adv. Eng. Softw..

[27]  Swagatam Das,et al.  A synergy of the sine-cosine algorithm and particle swarm optimizer for improved global optimization and object tracking , 2018, Swarm Evol. Comput..

[28]  Hamido Fujita,et al.  Efficient Robust Model Fitting for Multistructure Data Using Global Greedy Search , 2020, IEEE Transactions on Cybernetics.

[29]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[30]  Zong Woo Geem,et al.  A New Heuristic Optimization Algorithm: Harmony Search , 2001, Simul..

[31]  M. Fesanghary,et al.  An improved harmony search algorithm for solving optimization problems , 2007, Appl. Math. Comput..

[32]  Ling Wang,et al.  An effective co-evolutionary particle swarm optimization for constrained engineering design problems , 2007, Eng. Appl. Artif. Intell..

[33]  Dan Simon,et al.  Biogeography-Based Optimization , 2022 .

[34]  E. Sandgren,et al.  Nonlinear Integer and Discrete Programming in Mechanical Design Optimization , 1990 .

[35]  Jeng-Shyang Pan,et al.  Handwritten Arabic Manuscript Image Binarization Using Sine Cosine Optimization Algorithm , 2016, ICGEC.

[36]  Xu Chen,et al.  An opposition-based sine cosine approach with local search for parameter estimation of photovoltaic models , 2019, Energy Conversion and Management.

[37]  Carlos A. Coello Coello,et al.  Use of a self-adaptive penalty approach for engineering optimization problems , 2000 .

[38]  Ponnuthurai Nagaratnam Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2014 Special Session and Competition on Single Objective Real-Parameter Numerical Optimization , 2014 .

[39]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[40]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[41]  Ian C. Parmee,et al.  Evolutionary and adaptive computing in engineering design , 2001 .

[42]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[43]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[44]  Oguz Emrah Turgut,et al.  Thermal and Economical Optimization of a Shell and Tube Evaporator Using Hybrid Backtracking Search—Sine–Cosine Algorithm , 2017 .

[45]  MirjaliliSeyedali.  Moth-flame optimization algorithm , 2015 .

[46]  Kusum Deep,et al.  Hybrid sine cosine artificial bee colony algorithm for global optimization and image segmentation , 2019, Neural Computing and Applications.

[47]  Amir Hossein Gandomi,et al.  Cuckoo search algorithm: a metaheuristic approach to solve structural optimization problems , 2011, Engineering with Computers.

[48]  Hae Chang Gea,et al.  STRUCTURAL OPTIMIZATION USING A NEW LOCAL APPROXIMATION METHOD , 1996 .

[49]  Kusum Deep,et al.  Enhanced leadership-inspired grey wolf optimizer for global optimization problems , 2019, Engineering with Computers.

[50]  Hossam Faris,et al.  Salp Swarm Algorithm: A bio-inspired optimizer for engineering design problems , 2017, Adv. Eng. Softw..

[51]  Seyed Mohammad Mirjalili.  How effective is the Grey Wolf optimizer in training multi-layer perceptrons , 2014, Applied Intelligence.

[52]  Farid Nouioua,et al.  An improved sine cosine algorithm to select features for text categorization , 2020, J. King Saud Univ. Comput. Inf. Sci..

[53]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[54]  Kusum Deep,et al.  A hybrid self-adaptive sine cosine algorithm with opposition based learning , 2019, Expert Syst. Appl..

[55]  R. Venkata Rao,et al.  Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems , 2011, Comput. Aided Des..

[56]  K. Deb.  An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[57]  Jasbir S. Arora,et al.  Introduction to Optimum Design , 1988 .

[58]  Aboul Ella Hassanien,et al.  ASCA-PSO: Adaptive sine cosine optimization algorithm integrated with particle swarm for pairwise local sequence alignment , 2018, Expert Syst. Appl..

[59]  Mark A. Kramer,et al.  Improvement of the backpropagation algorithm for training neural networks , 1990 .

[60]  Carlos A. Coello Coello,et al.  Constraint-handling in genetic algorithms through the use of dominance-based tournament selection , 2002, Adv. Eng. Informatics.

[61]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[62]  Manuel López-Ibáñez,et al.  Ant colony optimization , 2010, GECCO '10.

[63]  Huiling Chen,et al.  Predicting Intentions of Students for Master Programs Using a Chaos-Induced Sine Cosine-Based Fuzzy K-Nearest Neighbor Classifier , 2019, IEEE Access.

[64]  Xiaoyong Liu,et al.  Parameter optimization of support vector regression based on sine cosine algorithm , 2018, Expert Syst. Appl..

[65]  Alireza Askarzadeh,et al.  A novel metaheuristic method for solving constrained engineering optimization problems: Crow search algorithm , 2016 .

[66]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[67]  Hany M. Hasanien,et al.  Optimal power flow solution in power systems using a novel Sine-Cosine algorithm , 2018, International Journal of Electrical Power & Energy Systems.

[68]  Kalyanmoy Deb,et al.  A combined genetic adaptive search (GeneAS) for engineering design , 1996 .

[69]  V. Braibant,et al.  Structural optimization: A new dual method using mixed variables , 1986 .

[70]  Ashok Dhondu Belegundu,et al.  A Study of Mathematical Programming Methods for Structural Optimization , 1985 .

[71]  Ali Rıza Yıldız,et al.  Comparison of grey wolf, whale, water cycle, ant lion and sine-cosine algorithms for the optimization of a vehicle engine connecting rod , 2018 .

[72]  Dinesh Gopalani,et al.  Opposition-Based Sine Cosine Algorithm (OSCA) for Training Feed-Forward Neural Networks , 2017, 2017 13th International Conference on Signal-Image Technology & Internet-Based Systems (SITIS).

[73]  R. M. Rizk-Allah,et al.  Hybridizing sine cosine algorithm with multi-orthogonal search strategy for engineering design problems , 2018, J. Comput. Des. Eng..

[74]  Ajoy Kumar Chakraborty,et al.  Solution of short-term hydrothermal scheduling using sine cosine algorithm , 2018, Soft Comput..

[75]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[76]  Singiresu S. Rao,et al.  Optimization Theory and Applications , 1980, IEEE Transactions on Systems, Man, and Cybernetics.

[77]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[78]  Vimal J. Savsani,et al.  Multi-objective sine-cosine algorithm (MO-SCA) for multi-objective engineering design problems , 2017, Neural Computing and Applications.

[79]  Mohammad Bagher Tavakoli,et al.  Modified Levenberg-Marquardt Method for Neural Networks Training , 2007 .

[80]  Dinesh Kumar,et al.  Data Clustering Using Sine Cosine Algorithm: Data Clustering Using SCA , 2017 .

[81]  Pengfei Duan,et al.  A Hybrid Method of Sine Cosine Algorithm and Differential Evolution for Feature Selection , 2017, ICONIP.

[82]  P. N. Suganthan,et al.  Problem Definitions and Evaluation Criteria for CEC 2011 Competition on Testing Evolutionary Algorithms on Real World Optimization Problems , 2011 .

[83]  Kusum Deep,et al.  Improved sine cosine algorithm with crossover scheme for global optimization , 2019, Knowl. Based Syst..

[84]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).