Domain Segmentation based on Uncertainty in the Surrogate (DSUS)

This paper develops a novel approach to characterize the uncertainty in the accuracy of surrogate models. This technique segregates the design domain based on the level of cross-validation errors; the overall framework is called Domain Segmentation based on Uncertainty in the Surrogate (DSUS). The estimated errors are classified into physically meaningful classes based on the user’s understanding of the system and/or the accuracy requirements for the concerned system analysis. In each class, the distribution of the cross-validation errors is estimated to represent the uncertainty in the surrogate. Support Vector Machine (SVM) is implemented to determine the boundaries between error classes, and to classify any new design (point) into a meaningful class. The DSUS framework is illustrated using two different surrogate modeling methods: (i) the Kriging method, and (ii) the Adaptive Hybrid Functions (AHF). We apply the DSUS framework to a series of standard problems and engineering problems. The results show that the DSUS framework can successfully classify the design domain and quantify the uncertainty (prediction errors) in surrogates. More than 90% of the test points could be accurately classified into its error class. In real life engineering design, where we use predictive models with different levels of fidelity, the knowledge of the level of error and uncertainty at any location inside the design space is uniquely helpful.

[1]  N. Zheng,et al.  Global Optimization of Stochastic Black-Box Systems via Sequential Kriging Meta-Models , 2006, J. Glob. Optim..

[2]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[3]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007, DAC 2006.

[4]  Achille Messac,et al.  Comprehensive Product Platform Planning (CP 3 ) Framework: Presenting a Generalized Product Family Model , 2010 .

[5]  Noel A Cressie,et al.  Statistics for Spatial Data. , 1992 .

[6]  Timothy W. Simpson,et al.  Analysis of support vector regression for approximation of complex engineering analyses , 2003, DAC 2003.

[7]  Jie Zhang,et al.  A Response Surface-Based Cost Model for Wind Farm Design , 2012 .

[8]  Achille Messac,et al.  An adaptive hybrid surrogate model , 2012, Structural and Multidisciplinary Optimization.

[9]  M. Zako,et al.  Structural optimization using Kriging approximation , 2003 .

[10]  Chih-Jen Lin,et al.  A comparison of methods for multiclass support vector machines , 2002, IEEE Trans. Neural Networks.

[11]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[12]  Achintya Haldar,et al.  Probability, Reliability and Statistical Methods in Engineering Design (Haldar, Mahadevan) , 1999 .

[13]  Raphael T. Haftka,et al.  Surrogate-based Analysis and Optimization , 2005 .

[14]  Daniel W. Apley,et al.  Objective–Oriented Sequential Sampling for Simulation Based Robust Design Considering Multiple Sources of Uncertainty , 2012, DAC 2012.

[15]  R. Haftka,et al.  Importing Uncertainty Estimates from One Surrogate to Another , 2009 .

[16]  R. L. Hardy.  Multiquadric equations of topography and other irregular surfaces , 1971 .

[17]  Achille Messac,et al.  SURROGATE MODELING OF COMPLEX SYSTEMS USING ADAPTIVE HYBRID FUNCTIONS , 2011, DAC 2011.

[18]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[19]  Andy J. Keane,et al.  Computational Approaches for Aerospace Design: The Pursuit of Excellence , 2005 .

[20]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[21]  Timothy W. Simpson,et al.  Metamodels for Computer-based Engineering Design: Survey and recommendations , 2001, Engineering with Computers.

[22]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[23]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[24]  M. Rais-Rohani,et al.  Ensemble of metamodels with optimized weight factors , 2008 .

[25]  A. Basudhar,et al.  Adaptive explicit decision functions for probabilistic design and optimization using support vector machines , 2008 .

[26]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[27]  Néstor V. Queipo,et al.  Toward an optimal ensemble of kernel-based approximations with engineering applications , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[28]  Jeremy J. Michalek,et al.  A Decomposed Gradient-Based Approach for Generalized Platform Selection and Variant Design in Product Family , 2008 .

[29]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[30]  S. Sathiya Keerthi,et al.  Which Is the Best Multiclass SVM Method? An Empirical Study , 2005, Multiple Classifier Systems.

[31]  Achille Messac,et al.  Developing a Non-gradient Based Mixed-Discrete Optimization Approach for Comprehensive Product Platform Planning (CP 3 ) , 2010 .

[32]  Salvador Pintos,et al.  An Optimization Methodology of Alkaline-Surfactant-Polymer Flooding Processes Using Field Scale Numerical Simulation and Multiple Surrogates , 2004 .

[33]  Luciano Castillo,et al.  Unrestricted wind farm layout optimization (UWFLO): Investigating key factors influencing the maximum power generation , 2012 .

[34]  G. Gary Wang,et al.  Review of Metamodeling Techniques in Support of Engineering Design Optimization , 2007 .

[35]  Marti A. Hearst.  Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[36]  Kamran Behdinan,et al.  Aircraft wing box optimization considering uncertainty in surrogate models , 2010 .

[37]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[38]  Timothy W. Simpson,et al.  Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How Far We Have Come - Or Not , 2008 .

[39]  Wei Chen,et al.  Multiresponse and Multistage Metamodeling Approach for Design Optimization , 2009 .

[40]  Michael S. Eldred,et al.  Formulations for Surrogate-Based Optimization Under Uncertainty , 2002 .

[41]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox, Version 2.0 , 2002 .

[42]  R. Haftka,et al.  Ensemble of surrogates , 2007 .

[43]  Wei Chen,et al.  A New Variable-Fidelity Optimization Framework Based on Model Fusion and Objective-Oriented Sequential Sampling , 2007, DAC 2007.

[44]  Daniel W. Apley,et al.  Understanding the Effects of Model Uncertainty in Robust Design With Computer Experiments , 2006 .

[45]  Achille Messac,et al.  Extended Radial Basis Functions: More Flexible and Effective Metamodeling , 2004 .

[46]  T. W. Layne,et al.  A Comparison of Approximation Modeling Techniques: Polynomial Versus Interpolating Models , 1998 .

[47]  Masoud Rais-Rohani,et al.  Ensemble of Metamodels with Optimized Weight Factors , 2008 .

[48]  Achille Messac,et al.  SDM 2011 Student Papers Competition A New Robust Surrogate Model: Reliability Based Hybrid Functions , 2011 .

[49]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[50]  V. Picheny.  Improving accuracy and compensating for uncertainty in surrogate modeling , 2009 .