Bayesian Adaptive Smoothing Spline using Stochastic Differential Equations

The smoothing spline is one of the most popular curve-fitting methods, partly because of empirical evidence supporting its effectiveness and partly because of its elegant mathematical formulation. However, there are two obstacles that restrict the use of smoothing spline in practical statistical work. Firstly, it becomes computationally prohibitive for large data sets because the number of basis functions roughly equals the sample size. Secondly, its global smoothing parameter can only provide constant amount of smoothing, which often results in poor performances when estimating inhomogeneous functions. In this work, we introduce a class of adaptive smoothing spline models that is derived by solving certain stochastic differential equations with finite element methods. The solution extends the smoothing parameter to a continuous data-driven function, which is able to capture the change of the smoothness of underlying process. The new model is Markovian, which makes Bayesian computation fast. A simulation study and real data example are presented to demonstrate the effectiveness of our method.

[1]  B. Ripley,et al.  Semiparametric Regression: Preface , 2003 .

[2]  Jun Yan,et al.  Gaussian Markov Random Fields: Theory and Applications , 2006 .

[3]  David Ruppert,et al.  Theory & Methods: Spatially‐adaptive Penalties for Spline Fitting , 2000 .

[4]  Jean Duchon,et al.  Splines minimizing rotation-invariant semi-norms in Sobolev spaces , 1976, Constructive Theory of Functions of Several Variables.

[5]  B. Mallick,et al.  Bayesian regression with multivariate linear splines , 2001 .

[6]  Adrian F. M. Smith,et al.  Automatic Bayesian curve fitting , 1998 .

[7]  Fabian Scheipl,et al.  Locally adaptive Bayesian P-splines with a Normal-Exponential-Gamma prior , 2009, Comput. Stat. Data Anal..

[8]  H. Rue,et al.  An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach , 2011 .

[9]  J. Staniswalis.  Local Bandwidth Selection for Kernel Estimates , 1989 .

[10]  S. Martino.  Approximate Bayesian Inference for Latent Gaussian Models , 2007 .

[11]  Alan Y. Chiang,et al.  Generalized Additive Models: An Introduction With R , 2007, Technometrics.

[12]  Ludwig Fahrmeir,et al.  Smoothing Hazard Functions and Time-Varying Effects in Discrete Duration and Competing Risks Models , 1996 .

[13]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[14]  Stig Larsson,et al.  Introduction to stochastic partial differential equations , 2008 .

[15]  R. Kass,et al.  Bayesian curve-fitting with free-knot splines , 2001 .

[16]  Yu Yue,et al.  Nonstationary Spatial Gaussian Markov Random Fields , 2010 .

[17]  H. Müller,et al.  Local Polynomial Modeling and Its Applications , 1998 .

[18]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[19]  G. Wahba,et al.  Hybrid Adaptive Splines , 1997 .

[20]  Veerabhadran Baladandayuthapani,et al.  Spatially Adaptive Bayesian Penalized Regression Splines (P-splines) , 2005 .

[21]  Andreas Brezger,et al.  Generalized structured additive regression based on Bayesian P-splines , 2006, Comput. Stat. Data Anal..

[22]  Leonhard Held,et al.  Gaussian Markov Random Fields: Theory and Applications , 2005 .

[23]  M. Wand,et al.  ON SEMIPARAMETRIC REGRESSION WITH O'SULLIVAN PENALIZED SPLINES , 2007 .

[24]  F. O’Sullivan.  A Statistical Perspective on Ill-posed Inverse Problems , 1986 .

[25]  Ludwig Fahrmeir,et al.  Function estimation with locally adaptive dynamic models , 2002, Comput. Stat..

[26]  Christopher Holmes,et al.  Spatially adaptive smoothing splines , 2006 .

[27]  Xiaotong Shen,et al.  Spatially Adaptive Regression Splines and Accurate Knot Selection Schemes , 2001 .

[28]  Paul H. C. Eilers,et al.  Flexible smoothing with B-splines and penalties , 1996 .

[29]  S. Lang,et al.  Bayesian P-Splines , 2004 .

[30]  B. Yandell,et al.  Locally adaptive smoothing splines , 1992 .

[31]  Robert Kohn,et al.  Locally Adaptive Nonparametric Binary Regression , 2007, 0709.3545.

[32]  M. Hansen,et al.  Spline Adaptation in Extended Linear Models , 1998 .

[33]  Sally Wood,et al.  Bayesian mixture of splines for spatially adaptive nonparametric regression , 2002 .

[34]  L. Fahrmeir,et al.  Multivariate statistical modelling based on generalized linear models , 1994 .

[35]  Felix Abramovich,et al.  Improved inference in nonparametric regression using Lk-smoothing splines , 1996 .

[36]  P. Speckman,et al.  Priors for Bayesian adaptive spline smoothing , 2012 .

[37]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[38]  Chong Gu.  Smoothing Spline Anova Models , 2002 .

[39]  Dongchu Sun,et al.  Fully Bayesian spline smoothing and intrinsic autoregressive priors , 2003 .

[40]  S. Wood.  Generalized Additive Models: An Introduction with R , 2006 .

[41]  Saad T. Bakir,et al.  Nonparametric Regression and Spline Smoothing , 2000, Technometrics.

[42]  G. Wahba.  Improper Priors, Spline Smoothing and the Problem of Guarding Against Model Errors in Regression , 1978 .

[43]  Ludwig Fahrmeir,et al.  Dynamic and semiparametric models , 2012 .

[44]  Paul H. C. Eilers,et al.  Splines, knots, and penalties , 2010 .

[45]  C. Ansley,et al.  The Signal Extraction Approach to Nonlinear Regression and Spline Smoothing , 1983 .

[46]  Douglas Nychka,et al.  Confidence Intervals for Nonparametric Curve Estimates , 2001 .

[47]  L. Fahrmeir,et al.  Bayesian inference for generalized additive mixed models based on Markov random field priors , 2001 .

[48]  G. Wahba,et al.  A Correspondence Between Bayesian Estimation on Stochastic Processes and Smoothing by Splines , 1970 .

[49]  C. Crainiceanu,et al.  Fast Adaptive Penalized Splines , 2008 .

[50]  G. Wahba.  Spline models for observational data , 1990 .