Penalized Estimation of Free-Knot Splines

Abstract Polynomial splines are often used in statistical regression models for smooth response functions. When the number and location of the knots are optimized, the approximating power of the spline is improved and the model is nonparametric with locally determined smoothness. However, finding the optimal knot locations is an historically difficult problem. We present a new estimation approach that improves computational properties by penalizing coalescing knots. The resulting estimator is easier to compute than the unpenalized estimates of knot positions, eliminates unnecessary “corners” in the fitted curve, and in simulation studies, shows no increase in the loss. A number of GCV and AIC type criteria for choosing the number of knots are evaluated via simulation.

[1]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[2]  Young K. Truong,et al.  Polynomial splines and their tensor products in extended linearmodeling , 1997 .

[3]  Young K. Truong,et al.  Polynomial splines and their tensor products in extended linear modeling: 1994 Wald memorial lecture , 1997 .

[4]  J. Friedman,et al.  FLEXIBLE PARSIMONIOUS SMOOTHING AND ADDITIVE MODELING , 1989 .

[5]  G. Wahba.  Spline models for observational data , 1990 .

[6]  G. Wahba,et al.  Hybrid Adaptive Splines , 1997 .

[7]  M. C. Jones,et al.  Spline Smoothing and Nonparametric Regression. , 1989 .

[8]  H. G. Burchard,et al.  Splines (with optimal knots) are better , 1974 .

[9]  Adrian F. M. Smith,et al.  Automatic Bayesian curve fitting , 1998 .

[10]  A. Gallant,et al.  Fitting Segmented Polynomial Regression Models Whose Join Points Have to Be Estimated , 1973 .

[11]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[12]  Tom Lyche,et al.  Knot Insertion and Deletion Algorithms for B-Spline Curves and Surfaces , 1992 .

[13]  C. D. Boor,et al.  Least Squares Cubic Spline Approximation, II - Variable Knots , 1968 .

[14]  D. Pregibon.  Logistic Regression Diagnostics , 1981 .

[15]  John Aitchison,et al.  The Statistical Analysis of Compositional Data , 1986 .

[16]  W. J. Studden,et al.  Asymptotic Integrated Mean Square Error Using Least Squares and Bias Minimizing Splines , 1980 .

[17]  P. Gill,et al.  Chapter III Constrained nonlinear programming , 1989 .

[18]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[19]  D. Jupp.  Approximation to Data by Splines with Free Knots , 1978 .

[20]  C. D. Boor,et al.  Least Squares Cubic Spline Approximation I | Fixed Knots , 1968 .

[21]  Paul H. C. Eilers,et al.  Flexible smoothing with B-splines and penalties , 1996 .

[22]  Robert Kohn,et al.  Additive nonparametric regression with autocorrelated errors , 1998 .

[23]  F. O’Sullivan.  A Statistical Perspective on Ill-posed Inverse Problems , 1986 .

[24]  W. Härdle.  Smoothing Techniques: With Implementation in S , 1991 .