Robust Optimization - A Comprehensive Survey

This paper reviews the state-of-the-art in robust design optimization ‐ the search for designs and solutions which are immune with respect to production tolerances, parameter drifts during operation time, model sensitivities and others. Starting with a short glimps of Taguchi’s robust design methodology, a detailed survey of approaches to robust optimization is presented. This includes a detailed discussion on how to account for design uncertainties and how to measure robustness (i.e., how to evaluate robustness). The main focus will be on the different approaches to perform robust optimization in practice including the methods of mathematical programming, deterministic nonlinear optimization, and direct search methods such as stochastic approximation and evolutionary computation. It discusses the strenghts and weaknesses of the different methods, thus, providing a basis for guiding the engineer to the most appropriate techniques. It also addresses performance aspects and test scenarios for direct robust optimization techniques.

[1]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[2]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.

[3]  Bernhard Sendhoff,et al.  Three dimensional evolutionary aerodynamic design optimization with CMA-ES , 2005, GECCO '05.

[4]  Shigeyoshi Tsutsui,et al.  Genetic algorithms with a robust solution searching scheme , 1997, IEEE Trans. Evol. Comput..

[5]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[6]  Andy J. Keane,et al.  Robust Design of Compressor Fan Blades Against Erosion , 2006 .

[7]  Thomas Bäck,et al.  Robust design of multilayer optical coatings by means of evolutionary algorithms , 1998, IEEE Trans. Evol. Comput..

[8]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[9]  Jürgen Branke,et al.  Creating Robust Solutions by Means of Evolutionary Algorithms , 1998, PPSN.

[10]  Enrico Gobbetti,et al.  Encyclopedia of Electrical and Electronics Engineering , 1999 .

[11]  Thomas J. Santner,et al.  Design and analysis of computer experiments , 1998 .

[12]  Harish Agarwal,et al.  Reliability Based Design Optimization: Formulations and Methodologies , 2004 .

[13]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[14]  Melvyn Sim,et al.  Tractable Approximations to Robust Conic Optimization Problems , 2006, Math. Program..

[15]  J. C. Helton,et al.  Uncertainty and sensitivity analysis in the presence of stochastic and subjective uncertainty , 1997 .

[16]  Xiaoping Du,et al.  A robust design method using variable transformation and Gauss–Hermite integration , 2006 .

[17]  S. Ranji Ranjithan,et al.  Chance-constrained genetic algorithms , 1999 .

[18]  James R. Wilson,et al.  A revised simplex search procedure for stochastic simulation response-surface optimization , 1998, WSC '98.

[19]  B. Youn,et al.  Possibility-Based Design Optimization Method for Design Problems With Both Statistical and Fuzzy Input Data , 2006 .

[20]  Bernhard Sendhoff,et al.  Functions with noise-induced multimodality: a test for evolutionary robust Optimization-properties and performance analysis , 2006, IEEE Transactions on Evolutionary Computation.

[21]  Hei Leung,et al.  Using Genetic Diversity to Achieve Sustainable Rice Disease Management. , 2003, Plant disease.

[22]  Yijun Wang,et al.  Methods for Robust Multidisciplinary Design , 2000 .

[23]  Shigeyoshi Tsutsui,et al.  A comparative study on the effects of adding perturbations to phenotypic parameters in genetic algorithms with a robust solution searching scheme , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[24]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[25]  Manolis Papadrakakis,et al.  Design optimization of steel structures considering uncertainties , 2005 .

[26]  Adrian Thompson,et al.  Evolutionary techniques for fault tolerance , 1996 .

[27]  Byung Chai Lee,et al.  Development of a simple and efficient method for robust optimization , 2002 .

[28]  Arkadi Nemirovski,et al.  Robust optimization – methodology and applications , 2002, Math. Program..

[29]  Achintya Haldar,et al.  Probability, Reliability and Statistical Methods in Engineering Design (Haldar, Mahadevan) , 1999 .

[30]  George J. Klir,et al.  Fuzzy sets, uncertainty and information , 1988 .

[31]  Z. Mourelatos,et al.  A Design Optimization Method Using Evidence Theory , 2006, DAC 2005.

[32]  Zissimos P. Mourelatos,et al.  A Methodology for Trading-Off Performance and Robustness Under Uncertainty , 2006, DAC 2005.

[33]  Marco Tomassini,et al.  Using Genetic Algorithms For Robust Optimization In Financial Applications , 1998 .

[34]  Hans-Georg Beyer,et al.  Actuator Noise in Recombinant Evolution Strategies on General Quadratic Fitness Models , 2004, GECCO.

[35]  Haoyu Wang,et al.  Safety Envelope for Load Tolerance and its Application to Fatigue Reliability Design , 2006 .

[36]  Melvyn Sim,et al.  Robust Discrete Optimization , 2003 .

[37]  Hans-Georg Beyer,et al.  A Comparison of Evolution Strategies with Other Direct Search Methods in the Presence of Noise , 2003, Comput. Optim. Appl..

[38]  Bernhard Sendhoff,et al.  Three Dimensional Aerodynamic Optimization for an Ultra-Low Aspect Ratio Transonic Turbine Stator Blade , 2005 .

[39]  Joshua D. Deaton,et al.  Reliability Based Topology Optimization using a Hybrid Cellular Automaton Algorithm , 2005 .

[40]  Lotfi A. Zadeh,et al.  Fuzzy Sets , 1996, Inf. Control..

[41]  Kemper Lewis,et al.  A comprehensive robust design approach for decision trade-offs in complex systems design , 2001 .

[42]  Y. Kanno,et al.  Evaluation and Maximization of Robustness of Trusses by using Semidefinite Programming , 2005 .

[43]  Thomas Bäck,et al.  Evolutionary computation: comments on the history and current state , 1997, IEEE Trans. Evol. Comput..

[44]  J. Douglas Faires,et al.  Numerical Analysis , 1981 .

[45]  J. Spall Stochastic Optimization, Stochastic Approximation and Simulated Annealing , 1999 .

[46]  Kyung K. Choi,et al.  Integration of Possibility-Based Optimization and Robust Design for Epistemic Uncertainty , 2007 .

[47]  In-Soung Jung,et al.  Robust shape optimization of electromechanical devices , 1999 .

[48]  Jeffrey W. Herrmann,et al.  A genetic algorithm for minimax optimization problems , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[49]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[50]  Sigrún Andradóttir,et al.  A review of simulation optimization techniques , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[51]  Jin-Su Kang,et al.  Robust economic optimization of process design under uncertainty , 2004 .

[52]  G. Geoffrey Vining,et al.  Taguchi's parameter design: a panel discussion , 1992 .

[53]  J. F. Sturm,et al.  Robust One Period Option Modelling , 2002 .

[54]  Lawrence L. Green,et al.  Probabilistic Methods for Uncertainty Propagation Applied to Aircraft Design , 2002 .

[55]  Kevin Otto,et al.  Manufacturing System Robustness Through Integrated Modeling , 2001 .

[56]  J. Gurland Multidimensional Gaussian Distributions (Kenneth S. Miller) , 1966 .

[57]  Wei Chen,et al.  Towards a Better Understanding of Modeling Feasibility Robustness in Engineering Design , 2000 .

[58]  Kees Roos,et al.  Robust Solutions of Uncertain Quadratic and Conic-Quadratic Problems , 2002, SIAM J. Optim..

[59]  Thomas Bartz-Beielstein,et al.  Experimental research in evolutionary computation , 2007, GECCO '07.

[60]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[61]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[62]  James C. Spall,et al.  AN OVERVIEW OF THE SIMULTANEOUS PERTURBATION METHOD FOR EFFICIENT OPTIMIZATION , 1998 .

[63]  Kai-Yew Lum,et al.  Max-min surrogate-assisted evolutionary algorithm for robust design , 2006, IEEE Transactions on Evolutionary Computation.

[64]  Ingo Rechenberg,et al.  Evolutionsstrategie '94 , 1994, Werkstatt Bionik und Evolutionstechnik.

[65]  Melvyn Sim,et al.  The Price of Robustness , 2004, Oper. Res..

[66]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[67]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[68]  Farrokh Mistree,et al.  Robust Design of Cellular Materials With Topological and Dimensional Imperfections , 2006, DAC 2005.

[69]  Timothy W. Simpson,et al.  On the Use of Statistics in Design and the Implications for Deterministic Computer Experiments , 1997 .

[70]  John E. Dennis,et al.  Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[71]  Paul J. Layzell,et al.  Evolution of Robustness in an Electronics Design , 2000, ICES.

[72]  Yuri M. Ermoliev Stochastic Quasigradient Methods , 2009, Encyclopedia of Optimization.

[73]  Panos Y. Papalambros,et al.  Monotonicity and active set strategies in probabilistic design optimization , 2005 .

[74]  Kyung K. Choi,et al.  The Performance Moment Integration Method for Reliability-Based Robust Design Optimization , 2004, DAC 2004.

[75]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[76]  Kwon-Hee Lee,et al.  Robust optimization considering tolerances of design variables , 2001 .

[77]  Susan L. Albin,et al.  Robust Optimization of Experimentally Derived Objective Functions , 2003 .

[78]  Wu Li,et al.  Options for Robust Airfoil Optimization under Uncertainty , 2002 .

[79]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[80]  Michael C. Fu,et al.  Optimization for Simulation: Theory vs. Practice , 2002 .

[81]  J. S. Ivey,et al.  Nelder-Mead simplex modifications for simulation optimization , 1996 .

[82]  Melvyn Sim,et al.  Robust linear optimization under general norms , 2004, Oper. Res. Lett..

[83]  C. T. Kelley,et al.  An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima , 1995, SIAM J. Optim..

[84]  Kenneth Sörensen,et al.  A genetic algorithm for robust schedules in a just-in-time environment , 2002 .

[85]  Michael W. Trosset,et al.  Taguchi and Robust Optimization , 1996 .

[86]  J. Buckley,et al.  An Introduction to Fuzzy Logic and Fuzzy Sets , 2002 .

[87]  Art B. Owen,et al.  9 Computer experiments , 1996, Design and analysis of experiments.

[88]  Xiaoping Du,et al.  The use of metamodeling techniques for optimization under uncertainty , 2001 .

[89]  Jun S. Liu,et al.  Monte Carlo strategies in scientific computing , 2001 .

[90]  C. T. Kelley,et al.  Superlinear Convergence and Implicit Filtering , 1999, SIAM J. Optim..

[91]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1992, Artificial Intelligence.

[92]  T. Ray Constrained robust optimal design using a multiobjective evolutionary algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[93]  Michael C. Fu,et al.  Feature Article: Optimization for simulation: Theory vs. Practice , 2002, INFORMS J. Comput..

[94]  Dimitris Bertsimas,et al.  A Robust Optimization Approach to Supply Chain Management , 2004, IPCO.

[95]  Wei Chen,et al.  Local approximation of the efficient frontier in robust design , 2000 .

[96]  Jos F. Sturm,et al.  Robust One-Period Option Hedging , 2006, Oper. Res..

[97]  Wei Chen,et al.  Methodology for Managing the Effect of Uncertainty in Simulation-Based Design , 2000 .

[98]  Donald R. Houser,et al.  A ROBUST OPTIMIZATION PROCEDURE WITH VARIATIONS ON DESIGN VARIABLES AND CONSTRAINTS , 1995 .

[99]  Jacek Marczyk,et al.  Stochastic multidisciplinary improvement - Beyond optimization , 2000 .

[100]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[101]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[102]  Horst Greiner Robust filter design by stochastic optimization , 1994, Other Conferences.

[103]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[105]  Melvyn Sim,et al.  Robust discrete optimization and network flows , 2003, Math. Program..

[106]  A. Neumaier,et al.  A grid algorithm for bound constrained optimization of noisy functions , 1995 .

[107]  Yuri Ermoliev,et al.  Numerical techniques for stochastic optimization , 1988 .

[108]  Marco Savoia,et al.  Structural reliability analysis through fuzzy number approach, with application to stability , 2002 .

[109]  Hans-Georg Beyer,et al.  On the Effects of Outliers on Evolutionary Optimization , 2003, IDEAL.

[110]  Wei Chen,et al.  Quality utility : a Compromise Programming approach to robust design , 1999 .

[111]  Kennetb A. De Genetic Algorithms Are NOT Function Optimizers , 1992 .

[112]  Tapabrata Ray,et al.  A surrogate assisted parallel multiobjective evolutionary algorithm for robust engineering design , 2006 .

[113]  M. Papadrakakis,et al.  Multi-objective design optimization using cascade evolutionary computations , 2005 .

[114]  Bernhard Sendhoff,et al.  On the Impact of Systematic Noise on the Evolutionary Optimization Performance—A Sphere Model Analysis , 2004, Genetic Programming and Evolvable Machines.

[115]  T. Ross Fuzzy Logic with Engineering Applications , 1994 .

[116]  Farrokh Mistree,et al.  Robust Design for Multiscale and Multidisciplinary Applications , 2006 .

[117]  R. C.,et al.  EFFECT OF RANDOM GEOMETRIC UNCERTAINTY ON THE COMPUTATIONAL DESIGN OF A 3-D FLEXIBLE WING , 2022 .

[118]  R. J. Hamilton,et al.  A technique for optimally designing fibre-reinforced laminated plates with manufacturing uncertainties for maximum buckling strength , 2005 .

[119]  K. Ishii,et al.  DESIGN OPTIMIZATION FOR ROBUSTNESS USING QUADRATURE FACTORIAL MODELS , 1998 .

[120]  Shabbir Ahmed,et al.  On robust optimization of two-stage systems , 2004, Math. Program..

[121]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[122]  Bernhard Sendhoff,et al.  Evolution Strategies for Robust Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[123]  Yann Le Quéré,et al.  Solving a robust maintenance scheduling problem at the french railways company , 2022 .

[124]  Farrokh Mistree,et al.  A procedure for robust design: Minimizing variations caused by noise factors and control factors , 1996 .

[125]  F. A. Campos,et al.  Robust solutions using fuzzy chance constraints , 2006 .

[126]  Kenneth Sörensen,et al.  Tabu Searching for Robust Solutions , 2001 .

[127]  H. Robbins A Stochastic Approximation Method , 1951 .

[128]  Farrokh Mistree,et al.  An Approach to Robust Design of Turbulent Convective Systems , 2006 .

[129]  Shigeyoshi Tsutsui,et al.  A Robust Solution Searching Scheme in Genetic Search , 1996, PPSN.

[130]  Genichi Taguchi,et al.  Quality Engineering through Design Optimization , 1989 .

[131]  Bernhard Sendhoff,et al.  Trusted Evolutionary Algorithm , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[132]  R. Rackwitz Reliability analysis—a review and some perspectives , 2001 .

[133]  J. Mulvey,et al.  Making a case for robust optimization models , 1997 .

[134]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[135]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[136]  Jay D. Martin,et al.  A Monte Carlo Method for Reliability-Based Design Opitmization , 2006 .

[137]  Gyung-Jin Park,et al.  Robust optimization of an automobile rearview mirror for vibration reduction , 2001 .

[138]  Emily M. Zechman,et al.  An evolutionary algorithm to generate alternatives (EAGA) for engineering optimization problems , 2004 .

[139]  Carl D. Sorensen,et al.  A general approach for robust optimal design , 1993 .

[140]  John E. Renaud,et al.  Reliability based design optimization using response surfaces in application to multidisciplinary systems , 2004 .

[141]  V. J. Torczoit,et al.  Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[142]  Bernhard Sendhoff,et al.  Trade-Off between Performance and Robustness: An Evolutionary Multiobjective Approach , 2003, EMO.

[143]  MengChu Zhou,et al.  A life cycle engineering approach to development of flexible manufacturing systems , 2003, IEEE Trans. Robotics Autom..

[144]  John M. Wilson,et al.  Introduction to Stochastic Programming , 1998, J. Oper. Res. Soc..

[145]  Kari Sentz,et al.  Combination of Evidence in Dempster-Shafer Theory , 2002 .

[146]  Manolis Papadrakakis,et al.  Reliability based optimization of steel frames under seismic loading conditions using evolutionary computation , 2004 .

[147]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[148]  Thomas Bartz-Beielstein,et al.  Experimental Research in Evolutionary Computation - The New Experimentalism , 2010, Natural Computing Series.

[149]  Bu-Sung Lee,et al.  Inverse multi-objective robust evolutionary design , 2006, Genetic Programming and Evolvable Machines.

[150]  Roger J.-B. Wets Modeling and solution strategies for unconstrained stochastic optimization problems , 1984, Ann. Oper. Res..

[151]  Jürgen Branke,et al.  Evolutionary Optimization in Dynamic Environments , 2001, Genetic Algorithms and Evolutionary Computation.

[152]  Scott Ferson,et al.  SAND 2002-0835 Unlimited Release Printed April 2002 Combination of Evidence in Dempster-Shafer Theory , 2002 .

[153]  Laurent El Ghaoui,et al.  Robust Solutions to Least-Squares Problems with Uncertain Data , 1997, SIAM J. Matrix Anal. Appl..

[154]  Qinghong Chen,et al.  Comparing Probabilistic and Fuzzy Set Approaches for Designing in the Presence of Uncertainty , 2000 .

[155]  Tae Hee Lee,et al.  Robust Design: An Overview , 2006 .

[156]  Yin Zhang General Robust-Optimization Formulation for Nonlinear Programming , 2007 .

[157]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[158]  Stephen Ekwaro-Osire,et al.  Life-cycle engineering: Issues, tools and research , 2003, Int. J. Comput. Integr. Manuf..

[159]  Timothy M. Cameron,et al.  Robust design optimization of structures through consideration of variation , 2002 .

[160]  I. Doltsinis,et al.  ROBUST DESIGN OF STRUCTURES USING OPTIMIZATION METHODS , 2004 .

[161]  Robert J. Vanderbei,et al.  Robust Optimization of Large-Scale Systems , 1995, Oper. Res..

[162]  Phil Husbands,et al.  A Comparison of Search Techniques on a Wing-Box Optimisation Problem , 1996, PPSN.

[163]  David B. Fogel,et al.  Evolving artificial intelligence , 1992 .

[164]  Zoran Filipi,et al.  Robust Optimization of an Automotive Valvetrain Using a Multiobjective Genetic Algorithm , 2003, DAC 2003.

[165]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[166]  T J Santner,et al.  Robust optimization of total joint replacements incorporating environmental variables. , 1999, Journal of biomechanical engineering.

[167]  Sankaran Mahadevan,et al.  Inclusion of Model Errors in Reliability-Based Optimization , 2006 .

[168]  Erik K. Antonsson,et al.  Design parameter selection in the presence of noise , 1994 .

[169]  Christopher I. Byrnes,et al.  ON THE STEADY-STATE BEHAVIOR OF FORCED NONLINEAR SYSTEMS , 2003 .

[170]  Indraneel Das,et al.  ROBUSTNESS OPTIMIZATION FOR CONSTRAINED NONLINEAR PROGRAMMING PROBLEMS , 2000 .

[171]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[172]  Panos Y. Papalambros,et al.  A Bayesian Approach to Reliability-Based Optimization With Incomplete Information , 2006, DAC 2006.

[173]  Arkadi Nemirovski,et al.  Robust Convex Optimization , 1998, Math. Oper. Res..

[174]  Michael C. Ferris,et al.  A Direct Search Algorithm for Optimization with Noisy Function Evaluations , 2000, SIAM J. Optim..

[175]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[176]  Xu Changwen,et al.  Fuzzy optimization of structures by the two-phase method , 1989 .