Global Optimization With Multivariate Adaptive Regression Splines

This paper presents a novel procedure for approximating the global optimum in structural design by combining multivariate adaptive regression splines (MARS) with a response surface methodology (RSM). MARS is a flexible regression technique that uses a modified recursive partitioning strategy to simplify high-dimensional problems into smaller yet highly accurate models. Combining MARS and RSM improves the conventional RSM by addressing highly nonlinear high-dimensional problems that can be simplified into lower dimensions, yet maintains a low computational cost and better interpretability when compared to neural networks and generalized additive models. MARS/RSM is also compared to simulated annealing and genetic algorithms in terms of computational efficiency and accuracy. The MARS/RSM procedure is applied to a set of low-dimensional test functions to demonstrate its convergence and limiting properties

[1]  Wei-Yen Wang,et al.  An online GA-based output-feedback direct adaptive fuzzy-neural controller for uncertain nonlinear systems , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  G. Kubin,et al.  A multi-band nonlinear oscillator model for speech , 1998, Conference Record of Thirty-Second Asilomar Conference on Signals, Systems and Computers (Cat. No.98CH36284).

[3]  H. Altay Güvenir,et al.  An overview of regression techniques for knowledge discovery , 1999, The Knowledge Engineering Review.

[4]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[5]  T. Simpson,et al.  Comparative studies of metamodelling techniques under multiple modelling criteria , 2001 .

[6]  Ching-Shiow Tseng,et al.  An orthogonal neural network for function approximation , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Chi-Chun Lo,et al.  A multiobjective hybrid genetic algorithm for the capacitated multipoint network design problem , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[8]  Shijin Ren,et al.  Modeling the Toxicity of Aromatic Compounds to Tetrahymena pyriformis: The Response Surface Methodology with Nonlinear Methods , 2003, J. Chem. Inf. Comput. Sci..

[9]  Nielen Stander,et al.  On the robustness of a simple domain reduction scheme for simulation‐based optimization , 2002 .

[10]  Kar-Ann Toh,et al.  Deterministic global optimization for FNN training , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[11]  T. Simpson,et al.  Analysis of support vector regression for approximation of complex engineering analyses , 2005, DAC 2003.

[12]  T. Simpson,et al.  Comparative studies of metamodeling techniques under multiple modeling criteria , 2000 .

[13]  Guoping Liu,et al.  Variable neural networks for adaptive control of nonlinear systems , 1999, IEEE Trans. Syst. Man Cybern. Part C.

[14]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[15]  Timothy W. Simpson,et al.  Sampling Strategies for Computer Experiments: Design and Analysis , 2001 .

[16]  Rainer Schwabe,et al.  EQUIDISTRIBUTED DESIGNS IN NONPARAMETRIC REGRESSION , 2003 .

[17]  Timothy W. Simpson,et al.  Analysis of support vector regression for approximation of complex engineering analyses , 2003, DAC 2003.

[18]  Andrew H. Sung,et al.  A comparative study of techniques for intrusion detection , 2003, Proceedings. 15th IEEE International Conference on Tools with Artificial Intelligence.

[19]  Alex Fukunaga,et al.  Automating the process of optimization in spacecraft design , 1997, 1997 IEEE Aerospace Conference.

[20]  Georges G. E. Gielen,et al.  Analysis of simulation-driven numerical performance modeling techniques for application to analog circuit optimization , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[21]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[22]  Jesús Muñoz,et al.  Comparison of statistical methods commonly used in predictive modelling , 2004 .

[23]  R. H. Myers,et al.  Response Surface Methodology: Process and Product Optimization Using Designed Experiments , 1995 .

[24]  S. V. Wunnava,et al.  Data encryption performance and evaluation schemes , 2002, Proceedings IEEE SoutheastCon 2002 (Cat. No.02CH37283).

[25]  Ken J. Craig,et al.  Automotive crashworthiness design using response surface‐based variable screening and optimization , 2005 .

[26]  Jerome Sacks,et al.  Designs for Computer Experiments , 1989 .

[27]  W. J. Whiten,et al.  Computational investigations of low-discrepancy sequences , 1997, TOMS.

[28]  Raphael T. Haftka,et al.  Response surface approximations for structural optimization , 1996 .

[29]  P. Lewis,et al.  Nonlinear Modeling of Time Series Using Multivariate Adaptive Regression Splines (MARS) , 1991 .

[30]  Gretchen G. Moisen,et al.  Comparing five modelling techniques for predicting forest characteristics , 2002 .

[31]  Qi-Jun Zhang,et al.  Optimal pin-assignment for ground noise minimization in IC packages and connectors , 1996, 1996 Proceedings 46th Electronic Components and Technology Conference.

[32]  Gerald W. Evans,et al.  Comparison of global search methods for design optimization using simulation , 1991, 1991 Winter Simulation Conference Proceedings..

[33]  Nielen Stander,et al.  The successive response surface method applied to sheet-metal forming , 2001 .

[34]  Wei Zha,et al.  A data mining approach to objective speech quality measurement , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[35]  Z. Michalewicz,et al.  A genetic algorithm for the linear transportation problem , 1991, IEEE Trans. Syst. Man Cybern..

[36]  K. O. Cogger.  Modern Regression Methods: A Comparative Discussion , 2001 .