Dynamic simulation metamodeling using MARS: A case of radar simulation

Dynamic system simulations require relating the inputs to the multivariate output which can be a function of time-space coordinates. In this work, we propose a methodology for the metamodeling of dynamic simulation models via Multivariate Adaptive Regression Splines (MARS). To handle incomplete output processes, where the simulation model does not produce an output in some steps due to missing inputs, we have devised a two-stage metamodeling scheme. The methodology is demonstrated on a dynamic radar simulation model. The prediction performance of the resulting metamodel is tested with four different sampling techniques (i.e., designs) and 16 sample sizes. We also investigate the effect of alternative coordinate system representations on the metamodeling performance. The results suggest that MARS is an effective method for metamodeling dynamic simulations, particularly, when expert judgment is not readily available. Results also show that there are interactions between the coordinate systems and sampling techniques, and some design-representation-size combinations are very promising in the metamodeling of radar simulations.

[1]  M. Stein.  Large sample properties of simulations using latin hypercube sampling , 1987 .

[2]  Xiaotong Shen,et al.  Spatially Adaptive Regression Splines and Accurate Knot Selection Schemes , 2001 .

[3]  Thomas J. Santner,et al.  The Design and Analysis of Computer Experiments , 2003, Springer Series in Statistics.

[4]  Young K. Truong,et al.  Polynomial splines and their tensor products in extended linear modeling: 1994 Wald memorial lecture , 1997 .

[5]  Peter C Austin,et al.  A comparison of regression trees, logistic regression, generalized additive models, and multivariate adaptive regression splines for predicting AMI mortality , 2007, Statistics in medicine.

[6]  Tao Chen,et al.  Efficient meta-modelling of complex process simulations with time-space-dependent outputs , 2011, Comput. Chem. Eng..

[7]  Jack P. C. Kleijnen,et al.  Sensitivity analysis of simulation experiments: regression analysis and statistical design , 1992 .

[8]  Tian-Shyug Lee,et al.  Mining the customer credit using classification and regression tree and multivariate adaptive regression splines , 2006, Comput. Stat. Data Anal..

[9]  Russell R. Barton,et al.  Chapter 18 Metamodel-Based Simulation Optimization , 2006, Simulation.

[10]  June F. D. Rodriguez,et al.  Metamodeling Techniques to Aid in the Aggregation Process of Large Hierarchical Simulation Models , 2012 .

[11]  P. Lewis,et al.  Nonlinear Modeling of Time Series Using Multivariate Adaptive Regression Splines (MARS) , 1991 .

[12]  Serhat Yesilyurt,et al.  Bayesian-validated computer-simulation surrogates for optimization and design: error estimates and applications , 1997 .

[13]  John E. Walsh,et al.  Assessing the response of area burned to changing climate in western boreal North America using a Multivariate Adaptive Regression Splines (MARS) approach , 2009 .

[14]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[15]  Tian-Shyug Lee,et al.  Sales forecasting for computer wholesalers: A comparison of multivariate adaptive regression splines and artificial neural networks , 2012, Decis. Support Syst..

[16]  G. Weber,et al.  CMARS: a new contribution to nonparametric regression with multivariate adaptive regression splines supported by continuous optimization , 2012 .

[17]  Les E. Atlas,et al.  Recurrent neural networks and robust time series prediction , 1994, IEEE Trans. Neural Networks.

[18]  Kai Virtanen,et al.  Analyzing air combat simulation results with dynamic bayesian networks , 2007, 2007 Winter Simulation Conference.

[19]  S. Rippa,et al.  Numerical Procedures for Surface Fitting of Scattered Data by Radial Functions , 1986 .

[20]  Adrian F. M. Smith,et al.  Automatic Bayesian curve fitting , 1998 .

[21]  Michael A. West,et al.  A dynamic modelling strategy for Bayesian computer model emulation , 2009 .

[22]  M. J. Bayarri,et al.  Computer model validation with functional output , 2007, 0711.3271.

[23]  Averill M. Law,et al.  Simulation Modeling and Analysis , 1982 .

[24]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[25]  Dimitri N. Mavris,et al.  Systems-of-Systems Analysis of Ballistic Missile Defense Architecture Effectiveness through Surrogate Modeling and Simulation , 2010, 2008 2nd Annual IEEE Systems Conference.

[26]  Lyle H. Ungar,et al.  A comparison of two nonparametric estimation schemes: MARS and neural networks , 1993 .

[27]  James K. Hall,et al.  Sensorcraft Mission Simulation Study , 2002 .

[28]  Keiji Kanazawa,et al.  A model for reasoning about persistence and causation , 1989 .

[29]  Andy J. Keane,et al.  Recent advances in surrogate-based optimization , 2009 .

[30]  Douglas C. Montgomery,et al.  Multiple response surface methods in computer simulation , 1977 .

[31]  Manuel P. Cuéllar,et al.  An Application of Non-Linear Programming to Train Recurrent Neural Networks in Time Series Prediction Problems , 2005, ICEIS.

[32]  Linda Weiser Friedman,et al.  The Simulation Metamodel , 1995 .

[33]  R. Kass,et al.  Bayesian curve-fitting with free-knot splines , 2001 .

[34]  Mohamed A. Zohdy,et al.  Nonlinear model-based dynamic recurrent neural network , 2001, Proceedings of the 44th IEEE 2001 Midwest Symposium on Circuits and Systems. MWSCAS 2001 (Cat. No.01CH37257).

[35]  T. J. Mitchell,et al.  Exploratory designs for computational experiments , 1995 .

[36]  Gerhard-Wilhelm Weber,et al.  Robust Regression Metamodelling of Complex Systems: The Case of Solid Rocket Motor Performance Metamodelling , 2012, Advances in Intelligent Modelling and Simulation.

[37]  M. D. McKay,et al.  A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , 2000 .

[38]  M. Zelen.  The Use of Group Divisible Designs for Confounded Asymmetrical Factorial Arrangements , 1958 .

[39]  Gerhard-Wilhelm Weber,et al.  Efficient adaptive regression spline algorithms based on mapping approach with a case study on finance , 2014, Journal of Global Optimization.

[40]  Gretchen G. Moisen,et al.  Comparing five modelling techniques for predicting forest characteristics , 2002 .

[41]  Jie Zhang,et al.  Prediction of polymer quality in batch polymerisation reactors using robust neural networks , 1998 .

[42]  A. O'Hagan,et al.  Bayesian emulation of complex multi-output and dynamic computer models , 2010 .

[43]  Chun-Chieh Yang,et al.  A multivariate adaptive regression splines model for simulation of pesticide transport in soils , 2003 .

[44]  Jack P. C. Kleijnen.  Design and Analysis of Simulation Experiments , 2007 .

[45]  Manolis Papadrakakis,et al.  Structural optimization using evolution strategies and neural networks , 1998 .

[46]  Trevor Hastie,et al.  Polynomial splines and their tensor products in extended linear modeling. Discussion and rejoinder , 1997 .

[47]  L J Eaves,et al.  Common Disease Analysis Using Multivariate Adaptive Regression Splines (MARS): Genetic Analysis Workshop 12 Simulated Sequence Data , 2001, Genetic epidemiology.

[48]  A. Prasad,et al.  Newer Classification and Regression Tree Techniques: Bagging and Random Forests for Ecological Prediction , 2006, Ecosystems.

[49]  Russell R. Barton,et al.  A review on design, modeling and applications of computer experiments , 2006 .

[50]  A. O'Hagan,et al.  Gaussian process emulation of dynamic computer codes , 2009 .

[51]  David Rogers,et al.  G/SPLINES: A Hybrid of Friedman's Multivariate Adaptive Regression Splines (MARS) Algorithm with Holland's Genetic Algorithm , 1991, ICGA.

[52]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[53]  Chandrasekhar Kambhampati,et al.  Efficient recurrent neural network incorporating a priori knowledge , 2000 .

[54]  Russell R. Barton,et al.  Simulation metamodels , 1998, 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274).

[55]  K. G. Roquemore.  Hybrid Designs for Quadratic Response Surfaces , 1976 .

[56]  A. O'Hagan,et al.  Bayesian calibration of computer models , 2001 .

[57]  G. Weber,et al.  RCMARS: Robustification of CMARS with different scenarios under polyhedral uncertainty set , 2011 .

[58]  Jack P. C. Kleijnen,et al.  Kriging Metamodeling in Simulation: A Review , 2007, Eur. J. Oper. Res..

[59]  Inci Batmaz,et al.  Small response surface designs for metamodel estimation , 2003, Eur. J. Oper. Res..

[60]  R. Mesiar,et al.  Aggregation operators: new trends and applications , 2002 .

[61]  Cem Iyigun,et al.  Restructuring forward step of MARS algorithm using a new knot selection procedure based on a mapping approach , 2014, J. Glob. Optim..

[62]  G. Wahba,et al.  Hybrid Adaptive Splines , 1997 .

[63]  Lee W. Schruben,et al.  An experimental procedure for simulation response surface model identification , 1987, CACM.

[64]  Douglas C. Montgomery,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[65]  Andrea Grosso,et al.  Finding maximin latin hypercube designs by Iterated Local Search heuristics , 2009, Eur. J. Oper. Res..

[66]  Sourabh Bhattacharya,et al.  A simulation approach to Bayesian emulation of complex dynamic computer models , 2007 .

[67]  Robert B. Gramacy,et al.  Ja n 20 08 Bayesian Treed Gaussian Process Models with an Application to Computer Modeling , 2009 .

[68]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[69]  Timothy W. Simpson,et al.  Sampling Strategies for Computer Experiments: Design and Analysis , 2001 .