Sparsity considerations for dependent variables

The aim of this paper is to provide a comprehensive introduc- tion for the study of `1-penalized estimators in the context of dependent observations. We define a general `1-penalized estimator for solving prob- lems of stochastic optimization. This estimator turns out to be the LASSO (Tib96) in the regression estimation setting. Powerful theoretical guaran- tees on the statistical performances of the LASSO were provided in recent papers, however, they usually only deal with the iid case. Here, we study this estimator under various dependence assumptions.

[1]  Victor Chernozhukov,et al.  High Dimensional Sparse Econometric Models: An Introduction , 2011, 1106.5242.

[2]  Gilles Stoltz,et al.  Inverse problems and high dimensional estimation , 2011 .

[3]  Internal.  How to: Applications , 2010 .

[4]  A. Tsybakov,et al.  SPADES AND MIXTURE MODELS , 2009, 0901.2044.

[5]  G. Grimmett,et al.  ELECTRONIC COMMUNICATIONS in PROBABILITY , 2010 .

[6]  O. Wintenberger.  Deviation inequalities for sums of weakly dependent time series , 2009, 0911.1682.

[7]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[8]  Mohamed Hebiri.  Quelques questions de sélection de variables autour de l'estimateur Lasso , 2009 .

[9]  T. Mikosch,et al.  Infinite variance stable limits for sums of dependent random variables , 2009 .

[10]  A. Belloni,et al.  L1-Penalized Quantile Regression in High Dimensional Sparse Models , 2009, 0904.2931.

[11]  V. Koltchinskii.  Sparsity in penalized empirical risk minimization , 2009 .

[12]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[13]  Sylvie Huet,et al.  Gaussian model selection with an unknown variance , 2007, math/0701250.

[14]  Emmanuel Barillot,et al.  Classification of arrayCGH data using fused SVM , 2008, ISMB.

[15]  Pierre Alquier,et al.  Density estimation with quadratic loss: a confidence intervals method , 2006, ESAIM: Probability and Statistics.

[16]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[17]  A. Tsybakov,et al.  Aggregation for Gaussian regression , 2007, 0710.3654.

[18]  P. Doukhan,et al.  Weak Dependence: With Examples and Applications , 2007 .

[19]  Michael H. Neumann,et al.  Probability and moment inequalities for sums of weakly dependent random variables, with applications , 2007 .

[20]  Florentina Bunea,et al.  Sparse Density Estimation with l1 Penalties , 2007, COLT.

[21]  H. Leeb,et al.  Sparse Estimators and the Oracle Property, or the Return of Hodges' Estimator , 2007, 0704.1466.

[22]  E. Candès,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[23]  N. U. Prabhu,et al.  Stochastic Processes and Their Applications , 1999 .

[24]  D. Hinkley.  Annals of Statistics , 2006 .

[25]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[26]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[27]  P. Massart,et al.  Gaussian model selection , 2001 .

[28]  P. Doukhan,et al.  A new weak dependence condition and applications to moment inequalities , 1999 .

[29]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[30]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[31]  S. Haberman.  Independence and Dependence , 1996 .

[32]  N. Fisher,et al.  Probability Inequalities for Sums of Bounded Random Variables , 1994 .

[33]  D. Donoho,et al.  Basis pursuit , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[34]  R. Fildes.  Journal of the Royal Statistical Society (B): Gary K. Grunwald, Adrian E. Raftery and Peter Guttorp, 1993, “Time series of continuous proportions”, 55, 103–116.☆ , 1993 .

[35]  C. Stein,et al.  Estimation with Quadratic Loss , 1992 .

[36]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[37]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[38]  R. Cox,et al.  Journal of the Royal Statistical Society B , 1972 .

[39]  M. Rosenblatt.  A CENTRAL LIMIT THEOREM AND A STRONG MIXING CONDITION. , 1956, Proceedings of the National Academy of Sciences of the United States of America.

[40]  J. Doob.  Stochastic processes , 1953 .

[41]  O. William.  Journal Of The American Statistical Association V-28 , 1932 .

[42]  H. P..  Annales de l'Institut Henri Poincaré , 1931, Nature.