MODEL SELECTION FOR (AUTO-)REGRESSION WITH DEPENDENT DATA

In this paper, we study the problem of non parametric estimation of an unknown regression function from dependent data with sub-Gaussian errors. As a particular case, we handle the autoregressive framework. For this purpose, we consider a collection of finite dimensional linear spaces (e.g. linear spaces spanned by wavelets or piecewise polynomials on a possibly irregular grid) and we estimate the regression function by a least-squares estimator built on a data driven selected linear space among the collection. This data driven choice is performed via the minimization of a penalized criterion akin to the Mallows' C p . We state non asymptotic risk bounds for our estimator in some -norm and we show that it is adaptive in the minimax sense over a large class of Besov balls of the form Bα,p,∞ (R) with p ≥ 1.

[1]  V. Volkonskii,et al.  Some Limit Theorems for Random Functions. II , 1959 .

[2]  A. Kolmogorov,et al.  On Strong Mixing Conditions for Stationary Gaussian Processes , 1960 .

[3]  I. A. Ibragimov.  On The Spectrum Of Stationary Gaussian Sequences Satisfying the Strong Mixing Condition I. Necessary Conditions , 1965 .

[4]  Quelques inégalités sur les martingales, d’après Dubins et Freedman , 1970 .

[5]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[6]  C. L. Mallows.  Some comments on C_p , 1973 .

[7]  H. Akaike.  A new look at the statistical model identification , 1974 .

[8]  R. Shibata.  Selection of the order of an autoregressive model by Akaike's information criterion , 1976 .

[9]  R. Shibata.  An optimal selection of regression variables , 1981 .

[10]  Ker-Chau Li,et al.  Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set , 1987 .

[11]  A Family of Asymptotically Optimal Methods for Choosing the Order of a Projective Regression Estimate , 1993 .

[12]  I. Daubechies,et al.  Wavelets on the Interval and Fast Wavelet Transforms , 1993 .

[13]  I. Daubechies.  Ten Lectures on Wavelets , 1992 .

[14]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[15]  P. Doukhan.  Mixing: Properties and Examples , 1994 .

[16]  E. Masry,et al.  Minimum complexity regression estimation with weakly dependent observations , 1996, Proceedings of 1994 Workshop on Information Theory and Statistics.

[17]  S. Geer.  Exponential Inequalities for Martingales, with Application to Maximum Likelihood Estimation for Counting Processes , 1995 .

[18]  C. Mallows.  More comments on C p , 1995 .

[19]  Stephen S. Wilson,et al.  Random iterative models , 1996 .

[20]  G. Lorentz,et al.  Constructive approximation : advanced problems , 1996 .

[21]  I. Johnstone,et al.  Minimax estimation via wavelet shrinkage , 1998 .

[22]  Michael H. Neumann,et al.  Regression-type inference in nonparametric autoregression , 1998 .

[23]  Critères d'ergodicité géométrique ou arithmétique de modèles linéaires pertubés à représentation markovienne , 1998 .

[24]  Dharmendra S. Modha,et al.  Memory-Universal Prediction of Stationary Random Processes , 1998, IEEE Trans. Inf. Theory.

[25]  P. Massart,et al.  Risk bounds for model selection via penalization , 1999 .

[26]  M. Hoffmann.  On nonparametric estimation in nonlinear AR(1)-models , 1999 .

[27]  Y. Baraud.  Model selection for regression on a fixed design , 2000 .

[28]  P. Massart,et al.  An Adaptive Compression Algorithm in Besov Spaces , 2000 .

[29]  Colin L. Mallows,et al.  Some Comments on Cp , 2000, Technometrics.

[30]  Y. Baraud,et al.  ADAPTIVE ESTIMATION IN AUTOREGRESSION OR β-MIXING REGRESSION VIA MODEL SELECTION By , 2001 .

[31]  Y. Baraud.  Model selection for regression on a random design , 2002 .