Adaptive sequential design for regression on multi-resolution bases

This work investigates the problem of construction of designs for estimation and discrimination between competing linear models. In our framework, the unknown signal is observed with the addition of a noise and only a few evaluations of the noisy signal are available. The model selection is performed in a multi-resolution setting. In this setting, the locations of discrete sequential D and A designs are precisely constraint in a small number of explicit points. Hence, an efficient stochastic algorithm can be constructed that alternately improves the design and the model. Several numerical experiments illustrate the efficiency of our method for regression. One can also use this algorithm as a preliminary step to build response surfaces for sensitivity analysis.

[1]  Bruce E. Hajek,et al.  Cooling Schedules for Optimal Annealing , 1988, Math. Oper. Res..

[2]  Armin Iske,et al.  Hierarchical Nonlinear Approximation for Experimental Design and Statistical Data Fitting , 2007, SIAM J. Sci. Comput..

[3]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[4]  Douglas P. Wiens,et al.  Minimax designs for approximately linear regression , 1992 .

[5]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[6]  R. K. Meyer,et al.  The Coordinate-Exchange Algorithm for Constructing Exact Optimal Experimental Designs , 1995 .

[7]  Gerard Kerkyacharian,et al.  Regression in random design and warped wavelets , 2004 .

[8]  D. Geman,et al.  Hierarchical testing designs for pattern recognition , 2005, math/0507421.

[9]  J. Marron,et al.  Extent to which least-squares cross-validation minimises integrated square error in nonparametric density estimation , 1987 .

[10]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[11]  Aixia Guo,et al.  Gene Selection for Cancer Classification using Support Vector Machines , 2014 .

[12]  D. Wiens,et al.  Integer-Valued, Minimax Robust Designs for Estimation and Extrapolation in Heteroscedastic, Approximately Linear Models , 2000 .

[13]  Artur Klinger,et al.  Inference in high dimensional generalized linear models based on soft thresholding , 2001 .

[14]  P. Lancaster.  Curve and surface fitting , 1986 .

[15]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[16]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[17]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[18]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[19]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[20]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[21]  Atanu Biswas,et al.  An efficient design for model discrimination and parameter estimation in linear models , 2002 .

[22]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[23]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[24]  G. Box,et al.  A Basis for the Selection of a Response Surface Design , 1959 .

[25]  M. Deaton,et al.  Response Surfaces: Designs and Analyses , 1989 .

[26]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Holger Dette,et al.  The Theory of Canonical Moments with Applications in Statistics, Probability, and Analysis , 1997 .

[28]  L. Pronzato.  Adaptive optimization and $D$-optimum experimental design , 2000 .

[29]  T. Lai,et al.  Least Squares Estimates in Stochastic Regression Models with Applications to Identification and Control of Dynamic Systems , 1982 .

[30]  Yi Liu,et al.  Exact Wavelet Designs for Discrimination , 2006 .

[31]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[32]  D. Wiens,et al.  On exact minimax wavelet designs obtained by simulated annealing , 2003 .

[33]  J. Kiefer,et al.  Optimum Designs in Regression Problems , 1959 .

[34]  Anestis Antoniadis,et al.  Wavelet kernel penalized estimation for non-equispaced design regression , 2006, Stat. Comput..