Nonlinear regression modeling via the lasso-type regularization

We consider the problem of constructing nonlinear regression models with Gaussian basis functions, using lasso regularization. Regularization with a lasso penalty is an advantageous in that it estimates some coefficients in linear regression models to be exactly zero. We propose imposing a weighted lasso penalty on a nonlinear regression model and thereby selecting the number of basis functions effectively. In order to select tuning parameters in the regularization method, we use a deviance information criterion proposed by Spiegelhalter et al. (2002), calculating the effective number of parameters by Gibbs sampling. Simulation results demonstrate that our methodology performs well in various situations.

[1]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[2]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[3]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[4]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[5]  Wenjiang J. Fu.  Penalized Regressions: The Bridge versus the Lasso , 1998 .

[6]  J. Friedman,et al.  A Statistical View of Some Chemometrics Regression Tools , 1993 .

[7]  G. Kitagawa,et al.  Generalised information criteria in model selection , 1996 .

[8]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[9]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[10]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[11]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[12]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[13]  S. Konishi,et al.  Nonlinear regression modeling via regularized radial basis function networks , 2008 .

[14]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[15]  Satoru Miyano,et al.  Weighted lasso in graphical Gaussian modeling for large gene network estimation based on microarray data. , 2007, Genome informatics. International Conference on Genome Informatics.

[16]  Trevor Hastie,et al.  The elements of statistical learning. 2001 , 2001 .

[17]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[18]  Seiya Imoto,et al.  Selection of smoothing parameters inB-spline nonparametric regression models using information criteria , 2003 .

[19]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[20]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[21]  G. Kitagawa,et al.  Information Criteria and Statistical Modeling , 2007 .

[22]  S. Konishi,et al.  Bayesian information criteria and smoothing parameter selection in radial basis function networks , 2004 .

[23]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .