A hybrid shuffled complex evolution approach with pattern search for unconstrained optimization

The difficulties associated with using classical mathematical programming methods on complex optimization problems have contributed to the development of alternative and efficient numerical approaches. Recently, to overcome the limitations of classical optimization methods, researchers have proposed a wide variety of meta-heuristics for searching near-optimum solutions to problems. Among the existing meta-heuristic algorithms, a relatively new optimization paradigm is the Shuffled Complex Evolution at the University of Arizona (SCE-UA) which is a global optimization strategy that combines concepts of the competition evolution theory, downhill simplex procedure of Nelder-Mead, controlled random search and complex shuffling. In an attempt to reduce processing time and improve the quality of solutions, particularly to avoid being trapped in local optima, in this paper is proposed a hybrid SCE-UA approach. The proposed hybrid algorithm is the combination of SCE-UA (without Nelder-Mead downhill simplex procedure) and a pattern search approach, called SCE-PS, for unconstrained optimization. Pattern search methods are derivative-free, meaning that they do not use explicit or approximate derivatives. Moreover, pattern search algorithms are direct search methods well suitable for the global optimization of highly nonlinear, multiparameter, and multimodal objective functions. The proposed SCE-PS method is tested with six benchmark optimization problems. Simulation results show that the proposed SCE-PS improves the searching performance when compared with the classical SCE-UA and a genetic algorithm with floating-point representation for all the tested problems. As evidenced by the performance indices based on the mean performance of objective function in 30 runs and mean of computational time, the SCE-PS algorithm has demonstrated to be effective and efficient at locating best-practice optimal solutions for unconstrained optimization.

[1]  Charles Audet,et al.  Analysis of Generalized Pattern Searches , 2000, SIAM J. Optim..

[2]  Soroosh Sorooshian,et al.  Calibration of rainfall‐runoff models: Application of global optimization to the Sacramento Soil Moisture Accounting Model , 1993 .

[3]  Charles Audet,et al.  A Pattern Search Filter Method for Nonlinear Programming without Derivatives , 2001, SIAM J. Optim..

[4]  Q. Duan,et al.  A global optimization strategy for efficient and effective calibration of hydrologic models. , 1991 .

[5]  M. H. Afshar,et al.  A parameter free Continuous Ant Colony Optimization Algorithm for the optimal design of storm sewer networks: Constrained and unconstrained approach , 2010, Adv. Eng. Softw..

[6]  Tiesong Hu,et al.  A Shuffled Complex Evolution of Particle Swarm Optimization Algorithm , 2007, ICANNGA.

[7]  J. Nicell,et al.  Evaluation of global optimization methods for conceptual rainfall-runoff model calibration , 1997 .

[8]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[9]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[10]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[11]  Charles H. Luce,et al.  Parameter identification for a runoff model for forest roads , 1994 .

[12]  S. Sorooshian,et al.  Effective and efficient global optimization for conceptual rainfall‐runoff models , 1992 .

[13]  Li Yu,et al.  Ant colony pattern search algorithms for unconstrained and bound constrained optimization , 2007, Appl. Math. Comput..

[14]  Francisco Herrera,et al.  Continuous scatter search: An analysis of the integration of some combination methods and improvement strategies , 2006, Eur. J. Oper. Res..

[15]  Jeffrey G. Arnold,et al.  Automatic calibration of a distributed catchment model , 2001 .

[16]  Mir-Bahador Aryanezhad,et al.  A new genetic algorithm for solving nonconvex nonlinear programming problems , 2008, Appl. Math. Comput..

[17]  T. Gan,et al.  Automatic Calibration of Conceptual Rainfall-Runoff Models: Optimization Algorithms, Catchment Conditions, and Model Structure , 1996 .

[18]  Qidi Wu,et al.  A novel ecological particle swarm optimization algorithm and its population dynamics analysis , 2008, Appl. Math. Comput..

[19]  S. Sorooshian,et al.  Shuffled complex evolution approach for effective and efficient global minimization , 1993 .

[20]  Robert Michael Lewis,et al.  Pattern Search Methods for Linearly Constrained Minimization , 1999, SIAM J. Optim..

[21]  Andries Petrus Engelbrecht,et al.  Bare bones differential evolution , 2009, Eur. J. Oper. Res..

[22]  W. Price.  Global optimization algorithms for a CAD workstation , 1987 .

[23]  Masao Fukushima,et al.  Tabu Search directed by direct search methods for nonlinear global optimization , 2006, Eur. J. Oper. Res..

[24]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[25]  John E. Dennis,et al.  Direct Search Methods on Parallel Machines , 1991, SIAM J. Optim..

[26]  Shu-Kai S. Fan,et al.  A hybrid simplex search and particle swarm optimization for unconstrained optimization , 2007, Eur. J. Oper. Res..

[27]  Yun-Wei Shang,et al.  A Note on the Extended Rosenbrock Function , 2006, Evolutionary Computation.

[28]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[29]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[30]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[31]  G. Kuczera.  Efficient subspace probabilistic parameter optimization for catchment models , 1997 .

[32]  A. K. Ziver,et al.  Estimation of distribution algorithms for nuclear reactor fuel management optimisation , 2006 .

[33]  M. J. Box.  A New Method of Constrained Optimization and a Comparison With Other Methods , 1965, Comput. J..

[34]  C. T. Kelley,et al.  Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition , 1999, SIAM J. Optim..

[35]  Virginia Torczon,et al.  On the Convergence of the Multidirectional Search Algorithm , 1991, SIAM J. Optim..

[36]  Ganapati Panda,et al.  Efficient prediction of stock market indices using adaptive bacterial foraging optimization (ABFO) and BFO based techniques , 2009, Expert Syst. Appl..

[37]  Masao Fukushima,et al.  Hybrid simulated annealing and direct search method for nonlinear unconstrained global optimization , 2002, Optim. Methods Softw..

[38]  J. A. Guin,et al.  Modification of the Complex Method of Constrained Optimization , 1968, Comput. J..

[39]  Robert Michael Lewis,et al.  Pattern Search Algorithms for Bound Constrained Minimization , 1999, SIAM J. Optim..

[40]  Kit Yan Chan,et al.  Improved orthogonal array based simulated annealing for design optimization , 2009, Expert Syst. Appl..

[41]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[42]  Zhi-Jia Li,et al.  APPLICATION OF GLOBAL OPTIMIZATION TECHNIQUE FOR CALIBRATING THE XINANJIANG WATERSHED MODEL , 2001 .

[43]  Robert Michael Lewis,et al.  A Globally Convergent Augmented Lagrangian Pattern Search Algorithm for Optimization with General Constraints and Simple Bounds , 2002, SIAM J. Optim..

[44]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..