Second-Order Experimental Designs for Simulation Metamodeling

The main purpose of this study is to compare the performance of a group of second-order designs such as Box-Behnken, face-center cube, three-level factorial, central composite, minimum bias, and minimum variance plus bias for estimating a quadratic metamodel. A time-shared computer system is used to demonstrate the ability of the designs in providing good fit of the metamodel to the simulation response. First, for various numbers of center runs, these designs are compared with respect to their efficiency, rotatability, orthogonality, robustness, bias, and prediction variance. Next, second-order metamodels are fit to the data collected using these designs. Metamodel fit is investigated using criteria such as average absolute error, PRESS, and the C p statistic. Results indicate that the minimum variance plus bias design is the most promising design to estimate a metamodel for the case studied.

[1]  James M. Lucas,et al.  Which Response Surface Design is Best: A Performance Comparison of Several Types of Quadratic Response Surface Designs in Symmetric Regions , 1976 .

[2]  H. Wynn.  Results in the Theory and Construction of D‐Optimum Experimental Designs , 1972 .

[3]  J. Kiefer,et al.  Optimum Designs in Regression Problems , 1959 .

[4]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[5]  H. Wynn.  The Sequential Generation of $D$-Optimum Experimental Designs , 1970 .

[6]  F. Pukelsheim.  Optimal Design of Experiments , 1993 .

[7]  Jack P. C. Kleijnen,et al.  Experimental design and regression analysis in simulation : An FMS case study , 1988 .

[8]  Andrej Pázman,et al.  Foundations of Optimum Experimental Design , 1986 .

[9]  Russell R. Barton,et al.  Metamodels for simulation input-output relations , 1992, WSC '92.

[10]  Toby J. Mitchell,et al.  Computer Construction of “D-Optimal” First-Order Designs , 1974 .

[11]  Randall P. Sadowski,et al.  Introduction to Simulation Using Siman , 1990 .

[12]  R. H. Myers.  Classical and modern regression with applications , 1986 .

[13]  R. C. St. John,et al.  D-Optimality for Regression Designs: A Review , 1975 .

[14]  Nripes Kumar Mandal.  On Robust Designs , 1989 .

[15]  E. Ziegel.  Optimal design and analysis of experiments , 1990 .

[16]  Jack P. C. Kleijnen,et al.  A methodology for fitting and validating metamodels in simulation , 2000, Eur. J. Oper. Res..

[17]  B. Yu,et al.  Metamodels in manufacturing: a review , 1994 .

[18]  Toby J. Mitchell,et al.  An Algorithm for the Construction of “D-Optimal” Experimental Designs , 2000, Technometrics.

[19]  J. Kiefer.  Optimum Experimental Designs , 1959 .

[20]  Colin L. Mallows,et al.  Some Comments on Cp , 2000, Technometrics.

[21]  Joan M. Donohue,et al.  Some optimal simulation designs for estimating quadratic response surface functions , 1990, 1990 Winter Simulation Conference Proceedings.

[22]  Raphael T. Haftka,et al.  Minimum-Bias Based Experimental Design for Constructing Response Surfaces in Structural Optimization , 1997 .

[23]  Norman R. Draper,et al.  Designs which minimize model inadequacies:Cuboidal regions of interest , 1965 .

[24]  Derek J. Pike,et al.  Empirical Model‐building and Response Surfaces. , 1988 .

[25]  Semra Tunali,et al.  Dealing with the least squares regression assumptions in simulation metamodeling , 2000 .

[26]  Anthony C. Atkinson,et al.  An adjustment algorithm for the construction of exact- D -optimum experimental designs , 1988 .

[27]  C. L. Mallows.  Some comments on C_p , 1973 .

[28]  Jack P. C. Kleijnen,et al.  Experimental Design for Sensitivity Analysis, Optimization and Validation of Simulation Models , 1997 .

[29]  Linda Weiser Friedman,et al.  The Simulation Metamodel , 1995 .

[30]  W. J. Studden,et al.  Optimal Experimental Designs , 1966 .

[31]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[32]  J. Kiefer,et al.  Two More Criteria Equivalent to D-Optimality of Designs , 1962 .

[33]  Russell R. Barton,et al.  METAMODELINC;: A STATE OF THE ART REVIEW , 1994 .

[34]  Christian N. Madu.  Simulation in manufacturing: A regression metamodel approach , 1990 .

[35]  M. Hossein Safizadeh,et al.  Minimizing the bias and variance of the gradient estimate in RSM simulation studies , 2002, Eur. J. Oper. Res..

[36]  Norman R. Draper,et al.  Center Points in Second-Order Response Surface Designs , 1982 .

[37]  A. J. Westlake.  Introduction to Linear Models and the Design and Analysis of Experiments , 1969 .

[38]  J. Kiefer,et al.  The Equivalence of Two Extremum Problems , 1960, Canadian Journal of Mathematics.

[39]  Stephen S. Lavenberg,et al.  Using conditional expectation to reduce variance in discrete event simulation , 1979, WSC '79.

[40]  N. Draper,et al.  Applied Regression Analysis. , 1967 .

[41]  Norman R. Draper,et al.  Another look at rotatability , 1990 .

[42]  C. Atwood.  Sequences Converging to $D$-Optimal Designs of Experiments , 1973 .

[43]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[44]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[45]  B. J. Winer.  Design and analysis of factorial experiments. , 1962 .

[46]  G. Geoffrey Vining,et al.  A Computer Program for Generating Variance Dispersion Graphs , 1993 .

[47]  James M. Lucas,et al.  Design Efficiencies for Varying Numbers of Centre Points , 1977 .

[48]  Joan M. Donohue,et al.  Simulation Designs and Correlation Induction for Reducing Second-Order Bias in First-Order Response Surfaces , 1993, Oper. Res..