A new hybrid optimization method combining artificial bee colony and limited-memory BFGS algorithms for efficient numerical optimization

Abstract In this paper, a new optimization method, which is developed especially for optimization of functions with a large number of local minima, is presented. The proposed method is a hybrid optimization algorithm which employs the artificial bee colony (ABC) and limited-memory Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) algorithms for combining their powerful features. The most prominent feature of the proposed method over other methods is that it provides accurate results and valuable convergence speeds, as well as easy implementation at the same time. Extensive simulation results supported by detailed statistical analyses show that the proposed method can be used for efficient optimization of functions including well-known benchmark functions and CEC2016 competition functions.

[1]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[2]  D. Karaboga,et al.  On the performance of artificial bee colony (ABC) algorithm , 2008, Appl. Soft Comput..

[3]  Amitava Chatterjee,et al.  MMSE design of nonlinear Volterra equalizers using artificial bee colony algorithm , 2013 .

[4]  Bin Wu,et al.  Hybrid harmony search and artificial bee colony algorithm for global optimization problems , 2012, Comput. Math. Appl..

[5]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[6]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[7]  Shen Pei-ping,et al.  Hybrid Artificial Bee Colony Algorithm and Particle Swarm Search for Global Optimization , 2014 .

[8]  Graham Kendall,et al.  An adaptive multi-population artificial bee colony algorithm for dynamic optimisation problems , 2016, Knowl. Based Syst..

[9]  Luca Vassio,et al.  A hybrid ABC for expensive optimizations: CEC 2016 competition benchmark , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[10]  Junjie Li,et al.  Artificial bee colony algorithm and pattern search hybridized for global optimization , 2013, Appl. Soft Comput..

[11]  Quoc V. Le,et al.  On optimization methods for deep learning , 2011, ICML.

[12]  Meng-Sing Liou,et al.  Adaptive directional local search strategy for hybrid evolutionary multiobjective optimization , 2014, Appl. Soft Comput..

[13]  Harikrishna Narasimhan,et al.  Parallel artificial bee colony (PABC) algorithm , 2009, 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC).

[14]  Alper Bastürk,et al.  Performance analysis of the coarse-grained parallel model of the artificial bee colony algorithm , 2013, Inf. Sci..

[15]  Chinta Sivadurgaprasad,et al.  Single phase multi-group teaching learning algorithm for computationally expensive numerical optimization (CEC 2016) , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[16]  Scott Kirkpatrick,et al.  Optimization by Simmulated Annealing , 1983, Sci..

[17]  Hai-Bin Duan,et al.  A Hybrid Artificial Bee Colony Optimization and Quantum Evolutionary Algorithm for Continuous Optimization Problems , 2010, Int. J. Neural Syst..

[18]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[19]  Tiranee Achalakul,et al.  The best-so-far selection in Artificial Bee Colony algorithm , 2011, Appl. Soft Comput..

[20]  Rafael Stubs Parpinelli,et al.  Parallel Approaches for the Artificial Bee Colony Algorithm , 2011 .

[21]  Narasimhan Sundararajan,et al.  Improved SRPSO algorithm for solving CEC 2015 computationally expensive numerical optimization problems , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[22]  Amitava Chatterjee,et al.  An artificial bee colony-least square algorithm for solving harmonic estimation problems , 2013, Appl. Soft Comput..

[23]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[24]  Dervis Karaboga,et al.  A modified Artificial Bee Colony (ABC) algorithm for constrained optimization problems , 2011, Appl. Soft Comput..

[25]  Dervis Karaboga,et al.  A quick artificial bee colony (qABC) algorithm and its performance on optimization problems , 2014, Appl. Soft Comput..

[26]  Bilal Alatas,et al.  Chaotic bee colony algorithms for global numerical optimization , 2010, Expert Syst. Appl..

[27]  Thomas Stützle,et al.  Improving Performance via Population Growth and Local Search: The Case of the Artificial Bee Colony Algorithm , 2011, Artificial Evolution.

[28]  Ponnuthurai Nagaratnam Suganthan,et al.  Problem Definitions and Evaluation Criteria for CEC 2015 Special Session on Bound Constrained Single-Objective Computationally Expensive Numerical Optimization , 2015 .

[29]  Dervis Karaboga,et al.  A comparative study of Artificial Bee Colony algorithm , 2009, Appl. Math. Comput..

[30]  W. Y. Szeto,et al.  An artificial bee colony algorithm for the capacitated vehicle routing problem , 2011, Eur. J. Oper. Res..

[31]  Bahriye Akay,et al.  Comparisons of metaheuristic algorithms and fitness functions on software test data generation , 2016, Appl. Soft Comput..

[32]  Dervis Karaboga,et al.  Artificial bee colony algorithm variants on constrained optimization , 2017 .

[33]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[34]  Jiong Shen,et al.  Automatic fuzzy partitioning approach using Variable string length Artificial Bee Colony (VABC) algorithm , 2012, Appl. Soft Comput..

[35]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[36]  Istvan Erlich,et al.  Solving the CEC2016 Real-Parameter Single Objective Optimization Problems through MVMO-PHM: Technical report , 2016 .

[37]  Mahamed G. H. Omran,et al.  Global-best harmony search , 2008, Appl. Math. Comput..

[38]  Vincent Berthier.  Experiments on the CEC 2015 expensive optimization testbed , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[39]  Guoqiang Li,et al.  Development and investigation of efficient artificial bee colony algorithm for numerical function optimization , 2012, Appl. Soft Comput..

[40]  Serdar Özyön,et al.  Solution to non-convex economic dispatch problem with valve point effects by incremental artificial bee colony with local search , 2013, Appl. Soft Comput..

[41]  Sam Kwong,et al.  Gbest-guided artificial bee colony algorithm for numerical function optimization , 2010, Appl. Math. Comput..

[42]  Kok Lay Teo,et al.  A hybrid approach to constrained global optimization , 2016, Appl. Soft Comput..

[43]  Selcuk Aslan,et al.  Discovery of conserved regions in DNA sequences by Artificial Bee Colony (ABC) algorithm based methods , 2018, Natural Computing.

[44]  Dervis Karaboga,et al.  A new emigrant creation strategy for parallel Artificial Bee Colony algorithm , 2015, 2015 9th International Conference on Electrical and Electronics Engineering (ELECO).

[45]  Dogan Aydin,et al.  Composite artificial bee colony algorithms: From component-based analysis to high-performing algorithms , 2015, Appl. Soft Comput..

[46]  Abdullah Al-Dujaili,et al.  HumanCog: A cognitive architecture for solving optimization problems , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[47]  Lingling Huang,et al.  A global best artificial bee colony algorithm for global optimization , 2012, J. Comput. Appl. Math..

[48]  Dervis Karaboga,et al.  An adaptive and hybrid artificial bee colony algorithm (aABC) for ANFIS training , 2016, Appl. Soft Comput..

[49]  Hasan Badem,et al.  A new efficient training strategy for deep neural networks by hybridization of artificial bee colony and limited-memory BFGS optimization algorithms , 2017, Neurocomputing.

[50]  Istvan Erlich,et al.  MVMO for bound constrained single-objective computationally expensive numerical optimization , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[51]  Junjie Li,et al.  Structural inverse analysis by hybrid simplex artificial bee colony algorithms , 2009 .

[52]  Milan Tuba,et al.  Parallelization of the artificial bee colony (ABC) algorithm , 2010 .

[53]  Anna Syberfeldt,et al.  Parameter tuned CMA-ES on the CEC'15 expensive problems , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[54]  Alper Bastürk,et al.  Parallel Implementation of Synchronous Type Artificial Bee Colony Algorithm for Global Optimization , 2012, J. Optim. Theory Appl..

[55]  Tianjun Liao,et al.  Artificial bee colony algorithm with dynamic population size to combined economic and emission dispatch problem , 2014 .

[56]  Weifeng Gao,et al.  A modified artificial bee colony algorithm , 2012, Comput. Oper. Res..

[57]  Dervis Karaboga,et al.  A modified Artificial Bee Colony algorithm for real-parameter optimization , 2012, Inf. Sci..

[58]  Selcuk Aslan,et al.  A new artificial bee colony algorithm to solve the multiple sequence alignment problem , 2016, Int. J. Data Min. Bioinform..

[59]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[60]  Selcuk Aslan,et al.  Best Supported Emigrant Creation for Parallel Implementation of Artificial Bee Colony Algorithm , 2016 .

[61]  Dervis Karaboga,et al.  On clarifying misconceptions when comparing variants of the Artificial Bee Colony Algorithm by offering a new implementation , 2015, Inf. Sci..

[62]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[63]  Lingling Huang,et al.  Enhanced artificial bee colony algorithm through differential evolution , 2016, Appl. Soft Comput..

[64]  D Karaboga,et al.  A discrete artificial bee colony algorithm for detecting transcription factor binding sites in DNA sequences. , 2016, Genetics and molecular research : GMR.

[65]  Martin Middendorf,et al.  Performance evaluation of artificial bee colony optimization and new selection schemes , 2011, Memetic Comput..

[66]  Varun Punnathanam,et al.  Reduced Yin-Yang-Pair optimization and its performance on the CEC 2016 expensive case , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[67]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[68]  Miguel A. Vega-Rodríguez,et al.  Hybrid multiobjective artificial bee colony for multiple sequence alignment , 2016, Appl. Soft Comput..

[69]  T.C.E. Cheng,et al.  A modified artificial bee colony algorithm for order acceptance in two-machine flow shops , 2013 .

[70]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[71]  Sanyang Liu,et al.  Improved artificial bee colony algorithm for global optimization , 2011 .

[72]  Prakash Kotecha,et al.  Simultaneous heat transfer search for computationally expensive numerical optimization , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).