Using recursive algorithms for the efficient identification of smoothing spline ANOVA models

In this paper we present a unified discussion of different approaches to the identification of smoothing spline analysis of variance (ANOVA) models: (i) the “classical” approach (in the line of Wahba in Spline Models for Observational Data, 1990; Gu in Smoothing Spline ANOVA Models, 2002; Storlie et al. in Stat. Sin., 2011) and (ii) the State-Dependent Regression (SDR) approach of Young in Nonlinear Dynamics and Statistics (2001). The latter is a nonparametric approach which is very similar to smoothing splines and kernel regression methods, but based on recursive filtering and smoothing estimation (the Kalman filter combined with fixed interval smoothing). We will show that SDR can be effectively combined with the “classical” approach to obtain a more accurate and efficient estimation of smoothing spline ANOVA models to be applied for emulation purposes. We will also show that such an approach can compare favorably with kriging.

[1]  C. Leser.  A Simple Method of Trend Construction , 1961 .

[2]  Fred C. Schweppe,et al.  Evaluation of likelihood functions for Gaussian signals , 1965, IEEE Trans. Inf. Theory.

[3]  Steven A. Orszag,et al.  CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS , 1978 .

[4]  R. Byrd,et al.  A stochastic framework for recursive computation of spline functions: Part II, smoothing splines , 1980 .

[5]  C. Ansley,et al.  The Signal Extraction Approach to Nonlinear Regression and Spline Smoothing , 1983 .

[6]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[7]  Peter C. Young,et al.  Recursive estimation and forecasting of non‐stationary time series , 1990 .

[8]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  E. Prescott,et al.  Postwar U.S. Business Cycles: An Empirical Investigation , 1997 .

[10]  I. Sobol,et al.  Sensitivity Measures, ANOVA-like Techniques and the Use of Bootstrap , 1997 .

[11]  Peter C. Young,et al.  Nonlinear and Nonstationary Signal Processing , 1998, Technometrics.

[12]  Peter C. Young,et al.  Recursive and en-bloc approaches to signal extraction , 1999 .

[13]  P. Young,et al.  Stochastic, Dynamic Modelling and Signal Processing: Time Variable and State Dependent Parameter Estimation , 2000 .

[14]  Peter C. Young,et al.  The Identification and Estimation of Nonlinear Stochastic Systems , 2001 .

[15]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[16]  Chong Gu.  Smoothing Spline Anova Models , 2002 .

[17]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox , 2002 .

[18]  Søren Nymand Lophaven,et al.  DACE - A Matlab Kriging Toolbox, Version 2.0 , 2002 .

[19]  Hao Helen Zhang,et al.  Component selection and smoothing in smoothing spline analysis of variance models -- COSSO , 2003 .

[20]  Hao Helen Zhang,et al.  Component selection and smoothing in multivariate nonparametric regression , 2006, math/0702659.

[21]  Peter C. Young,et al.  State Dependent Parameter metamodelling and sensitivity analysis , 2007, Comput. Phys. Commun..

[22]  Jon C. Helton,et al.  Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models , 2009, Reliab. Eng. Syst. Saf..

[23]  Tobias Wagner,et al.  Modelling of a thermomechanically coupled forming process based on functional outputs from a finite element analysis and from experimental measurements , 2010 .

[24]  D. Steinberg,et al.  Computer experiments: a review , 2010 .

[25]  Brian J Reich,et al.  Surface Estimation, Variable Selection, and the Nonparametric Oracle Property. , 2011, Statistica Sinica.

[26]  Alistair I. Mees.  Nonlinear Dynamics and Statistics , 2012 .