Adaptive sequential design for regression on Schauder Basis

We present a new sequential algorithm to build both optimal design and model selection in a multi-resolution family of functions. This algorithm relies on a localization property of discrete sequential D and A-optimal designs for Schauder Basis. We use these property with a simulated annealing strategy to obtain our stochastic algorithm. We illustrate its efficiency on several numerical experiments.

[1]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[2]  R. K. Meyer,et al.  The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs , 1995 .

[3]  Holger Dette,et al.  The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis , 1997 .

[4]  M. Deaton,et al.  Response Surfaces: Designs and Analyses , 1989 .

[5]  T. Lai,et al.  Least Squares Estimates in Stochastic Regression Models with Applications to Identification and Control of Dynamic Systems , 1982 .

[6]  L. Pronzato.  Adaptive optimization and $D$-optimum experimental design , 2000 .

[7]  J. Kiefer,et al.  Optimum Designs in Regression Problems , 1959 .

[8]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[10]  E. Hlawka.  Funktionen von beschränkter Variatiou in der Theorie der Gleichverteilung , 1961 .

[11]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[12]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[13]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[14]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[15]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  G. Box,et al.  A Basis for the Selection of a Response Surface Design , 1959 .

[17]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[18]  Jerome H. Friedman.  Multivariate adaptive regression splines (with discussion) , 1991 .