Obstacles and difficulties for robust benchmark problems: A novel penalty-based robust optimisation method

This paper first identifies a substantial gap in the literature of robust optimisation relating to the simplicity, low-dimensionality, lack of bias, lack of deceptiveness, and lack of multi-modality of test problems. Five obstacles and difficulties such as desired number of variables, bias, deceptiveness, multi-modality, and flatness are then proposed to design challenging robust test problems and resolve the deficiency. A standard test suit of eight robust benchmark problems is proposed along with controlling parameters that allow researchers to adjust and achieve the desired level of difficulty. After the theoretical analysis of each proposed test function, a robust particle swarm optimisation (RPSO) algorithm and a robust genetic algorithm (RGA) are employed to investigate their effectiveness experimentally. The paper also inspects the effects of the proposed controlling parameters on the difficulty of the test problems and the proposal of a novel penalty function to penalize the solutions proportional to their sensitivity to perturbations in parameters. The results demonstrate that the proposed test problems are able to benchmark the performance of robust algorithm effectively and provide different, controllable levels of difficulty. In addition, the comparative results reveal the superior performance and merits of the proposed penalty-based method in finding robust solutions. Note: The source codes of the proposed robust test functions are publicity available at www.alimirjalili.com/RO.html.

[1]  Zeshui Xu,et al.  Direct clustering analysis based on intuitionistic fuzzy implication , 2014, Appl. Soft Comput..

[2]  Shigeyoshi Tsutsui,et al.  Genetic algorithms with a robust solution searching scheme , 1997, IEEE Trans. Evol. Comput..

[3]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[4]  Seyed Mohammad Mirjalili,et al.  Multi-Verse Optimizer: a nature-inspired algorithm for global optimization , 2015, Neural Computing and Applications.

[5]  Amir Hossein Alavi,et al.  An effective krill herd algorithm with migration operator in biogeography-based optimization , 2014 .

[6]  Seyed Mohammad Mirjalili,et al.  Chaotic krill herd optimization algorithm , 2014 .

[7]  Rajkumar Roy,et al.  Variable Dependence Interaction And Multi-objective Optimisation , 2002, GECCO.

[8]  Seyedali Mirjalili Shifted robust multi-objective test problems , 2015 .

[9]  L. Darrell Whitley,et al.  Evaluating Evolutionary Algorithms , 1996, Artif. Intell..

[10]  Andrew Lewis,et al.  Biogeography-based optimisation with chaos , 2014, Neural Computing and Applications.

[11]  Andrew Lewis,et al.  Adaptive gbest-guided gravitational search algorithm , 2014, Neural Computing and Applications.

[12]  Bu-Sung Lee,et al.  Inverse multi-objective robust evolutionary design optimization in the presence of uncertainty , 2005, GECCO '05.

[13]  Yaochu Jin,et al.  A social learning particle swarm optimization algorithm for scalable optimization , 2015, Inf. Sci..

[14]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[15]  Qun Lin,et al.  A Novel Adaptive ε -Constrained Method for Constrained Problem , 2015 .

[16]  Tetsuyuki Takahama,et al.  Constrained Optimization by the ε Constrained Differential Evolution with Gradient-Based Mutation and Feasible Elites , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[17]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[18]  Andrew Lewis,et al.  A tri-objective Particle Swarm Optimizer for designing line defect Photonic Crystal Waveguides , 2014 .

[19]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[20]  Amir Hossein Gandomi,et al.  Hybridizing harmony search algorithm with cuckoo search for global numerical optimization , 2014, Soft Computing.

[21]  Andrew Lewis,et al.  Novel frameworks for creating robust multi-objective benchmark problems , 2015, Inf. Sci..

[22]  T. Ray Constrained robust optimal design using a multiobjective evolutionary algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[23]  Xin Yao,et al.  Search biases in constrained evolutionary optimization , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[24]  Jürgen Branke,et al.  Efficient search for robust solutions by means of evolutionary algorithms and fitness approximation , 2006, IEEE Transactions on Evolutionary Computation.

[25]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[26]  Seyedali Mirjalili,et al.  Dragonfly algorithm: a new meta-heuristic optimization technique for solving single-objective, discrete, and multi-objective problems , 2015, Neural Computing and Applications.

[27]  Seyed Mohammad Mirjalili,et al.  Ions motion algorithm for solving optimization problems , 2015, Appl. Soft Comput..

[28]  J. Branke Reducing the sampling variance when searching for robust solutions , 2001 .

[29]  Kalyanmoy Deb,et al.  Introducing Robustness in Multi-Objective Optimization , 2006, Evolutionary Computation.

[30]  Andrew Lewis,et al.  Autonomous Particles Groups for Particle Swarm Optimization , 2014 .

[31]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[32]  Shapour Azarm,et al.  A multi-objective genetic algorithm for robust design optimization , 2005, GECCO '05.

[33]  Carlos A. Coello Coello,et al.  Modified Differential Evolution for Constrained Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[34]  António Gaspar-Cunha,et al.  Robustness in multi-objective optimization using evolutionary algorithms , 2008, Comput. Optim. Appl..

[35]  Siti Mariyam Hj. Shamsuddin,et al.  CAPSO: Centripetal accelerated particle swarm optimization , 2014, Inf. Sci..

[36]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[37]  Thomas Bäck,et al.  Robust Design of Multilayer Optical Coatings by Means of Evolution Strategies , 1998 .

[38]  A. Kai Qin,et al.  Self-adaptive differential evolution algorithm for numerical optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[39]  Seyed Mohammad Mirjalili,et al.  The Ant Lion Optimizer , 2015, Adv. Eng. Softw..

[40]  Andrew Lewis,et al.  Let a biogeography-based optimizer train your Multi-Layer Perceptron , 2014, Inf. Sci..

[41]  Kalyanmoy Deb,et al.  Searching for Robust Pareto-Optimal Solutions in Multi-objective Optimization , 2005, EMO.

[42]  Amir Hossein Gandomi,et al.  Stud krill herd algorithm , 2014, Neurocomputing.

[43]  Andrew Lewis,et al.  Confidence measure: A novel metric for robust meta-heuristic optimisation algorithms , 2015, Inf. Sci..

[44]  Bernhard Sendhoff,et al.  Evolutionary Algorithms in the Presence of Noise: To Sample or Not to Sample , 2007, 2007 IEEE Symposium on Foundations of Computational Intelligence.

[45]  Xin-She Yang Test Problems in Optimization , 2010, 1008.0549.

[46]  Bernhard Sendhoff,et al.  Robust Optimization - A Comprehensive Survey , 2007 .

[47]  Amir Hossein Gandomi,et al.  Chaotic Krill Herd algorithm , 2014, Inf. Sci..

[48]  Jürgen Branke,et al.  Creating Robust Solutions by Means of Evolutionary Algorithms , 1998, PPSN.

[49]  Andrew Lewis,et al.  Grey Wolf Optimizer , 2014, Adv. Eng. Softw..

[50]  Jing J. Liang,et al.  Dynamic multi-swarm particle swarm optimizer , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[51]  Ponnuthurai N. Suganthan,et al.  Efficient constraint handling for optimal reactive power dispatch problems , 2012, Swarm Evol. Comput..

[52]  Amir Hossein Gandomi,et al.  Hybrid krill herd algorithm with differential evolution for global numerical optimization , 2014, Neural Computing and Applications.

[53]  Pratyusha Rakshit,et al.  Extending multi-objective differential evolution for optimization in presence of noise , 2015, Inf. Sci..

[54]  N. Hansen,et al.  Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem , 2015, Evolutionary Computation.

[55]  Seyed Mohammad Mirjalili How effective is the Grey Wolf optimizer in training multi-layer perceptrons , 2014, Applied Intelligence.

[56]  Andrew Lewis,et al.  S-shaped versus V-shaped transfer functions for binary Particle Swarm Optimization , 2013, Swarm Evol. Comput..

[57]  Thomas Bäck,et al.  An Archive Maintenance Scheme for Finding Robust Solutions , 2010, PPSN.

[58]  Lothar Thiele,et al.  Comparison of Multiobjective Evolutionary Algorithms: Empirical Results , 2000, Evolutionary Computation.

[59]  Qidi Wu,et al.  An analysis of the migration rates for biogeography-based optimization , 2014, Inf. Sci..

[60]  Jun Liu,et al.  Consensus analysis for a class of stochastic PSO algorithm , 2014, Appl. Soft Comput..

[61]  Shahryar Rahnamayan,et al.  Metaheuristics in large-scale global continues optimization: A survey , 2015, Inf. Sci..

[62]  Kalyanmoy Deb,et al.  Multi-objective test problems, linkages, and evolutionary methodologies , 2006, GECCO.

[63]  Kalyanmoy Deb,et al.  Multimodal Deceptive Functions , 1993, Complex Syst..

[64]  K. Deb An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[65]  Carlos A. Coello Coello,et al.  Simple Feasibility Rules and Differential Evolution for Constrained Optimization , 2004, MICAI.

[66]  Thomas Bäck,et al.  Exploiting Overlap When Searching for Robust Optima , 2010, PPSN.

[67]  Qingfu Zhang,et al.  Multiobjective evolutionary algorithms: A survey of the state of the art , 2011, Swarm Evol. Comput..

[68]  A. Gandomi,et al.  A novel improved accelerated particle swarm optimization algorithm for global numerical optimization , 2014 .

[69]  R. Lyndon While,et al.  A review of multiobjective test problems and a scalable test problem toolkit , 2006, IEEE Transactions on Evolutionary Computation.

[70]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[71]  Andrew Lewis,et al.  A comparison of multi-objective optimisation metaheuristics on the 2D airfoil design problem , 2013 .

[72]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[73]  T.H.W. Bäck,et al.  Evolution Strategies for Robust Optimization , 2008 .

[74]  Konstantinos G. Margaritis,et al.  On benchmarking functions for genetic algorithms , 2001, Int. J. Comput. Math..

[75]  L. Darrell Whitley,et al.  Fundamental Principles of Deception in Genetic Search , 1990, FOGA.

[76]  Bernhard Sendhoff,et al.  Trade-Off between Performance and Robustness: An Evolutionary Multiobjective Approach , 2003, EMO.

[77]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[78]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[79]  Jing J. Liang,et al.  Novel composition test functions for numerical global optimization , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[80]  Andrew Lewis,et al.  Novel performance metrics for robust multi-objective optimization algorithms , 2015, Swarm Evol. Comput..

[81]  Carlos A. Coello Coello,et al.  Constraint-handling in nature-inspired numerical optimization: Past, present and future , 2011, Swarm Evol. Comput..

[82]  António Gaspar-Cunha,et al.  Evolutionary robustness analysis for multi-objective optimization: benchmark problems , 2013, Structural and Multidisciplinary Optimization.

[83]  Thomas Bäck,et al.  A Survey of Evolution Strategies , 1991, ICGA.

[84]  Thomas Bäck,et al.  Robust design of multilayer optical coatings by means of evolutionary algorithms , 1998, IEEE Trans. Evol. Comput..

[85]  Patrick Siarry,et al.  A survey on optimization metaheuristics , 2013, Inf. Sci..

[86]  Kalyanmoy Deb,et al.  Multi-objective Genetic Algorithms: Problem Difficulties and Construction of Test Problems , 1999, Evolutionary Computation.

[87]  Qiang Long,et al.  A constraint handling technique for constrained multi-objective genetic algorithm , 2014, Swarm Evol. Comput..

[88]  P. Suganthan Particle swarm optimiser with neighbourhood operator , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[89]  Tamer Ölmez,et al.  A new metaheuristic for numerical function optimization: Vortex Search algorithm , 2015, Inf. Sci..

[90]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[91]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[92]  Alberto Lovison,et al.  A Polynomial Chaos Approach to Robust Multiobjective Optimization , 2009, Hybrid and Robust Approaches to Multiobjective Optimization.

[93]  S. Azarm,et al.  Multi-objective robust optimization using a sensitivity region concept , 2005 .

[94]  Bernhard Sendhoff,et al.  Evolution Strategies for Robust Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[95]  G. G. Wang,et al.  Design Space Reduction for Multi-Objective Optimization and Robust Design Optimization Problems , 2004 .