Weak transport inequalities and applications to exponential inequalities and oracle inequalities

We extend the dimension free Talagrand inequalities for convex distance \cite{talagrand:1995} using an extension of Marton's weak transport \cite{marton:1996a} to other metrics than the Hamming distance. We study the dual form of these weak transport inequalities for the euclidian norm and prove that it implies sub-gaussianity and convex Poincare inequality \cite{bobkov:gotze:1999a}. We obtain new weak transport inequalities for non products measures extending the results of Samson in \cite{samson:2000}. Many examples are provided to show that the euclidian norm is an appropriate metric for classical time series. Our approach, based on trajectories coupling, is more efficient to obtain dimension free concentration than existing contractive assumptions \cite{djellout:guillin:wu:2004,marton:2004}. Expressing the concentration properties of the ordinary least square estimator as a conditional weak transport problem, we derive new oracle inequalities with fast rates of convergence in dependent settings.

[1]  P. Doukhan,et al.  WEAKLY DEPENDENT CHAINS WITH INFINITE MEMORY , 2007, 0712.3231.

[2]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[3]  M. Ledoux.  On Talagrand's deviation inequalities for product measures , 1997 .

[4]  E. Rio.  Processus empiriques absolument réguliers et entropie universelle , 1998 .

[5]  Stephen S. Wilson,et al.  Random iterative models , 1996 .

[6]  C. Villani.  Optimal Transport: Old and New , 2008 .

[7]  Ludger Riischendorf.  The Wasserstein distance and approximation theorems , 1985 .

[8]  I. Ibragimov,et al.  Some Limit Theorems for Stationary Processes , 1962 .

[9]  Donald W. K. Andrews,et al.  Non-strong mixing autoregressive processes , 1984, Journal of Applied Probability.

[10]  Y. Ollivier,et al.  CURVATURE, CONCENTRATION AND ERROR ESTIMATES FOR MARKOV CHAIN MONTE CARLO , 2009, 0904.1312.

[11]  P. Bartlett,et al.  Empirical minimization , 2006 .

[12]  K. Ramanan,et al.  Concentration Inequalities for Dependent Random Variables via the Martingale Method , 2006, math/0609835.

[13]  K. Marton.  Measure concentration for Euclidean distance in the case of dependent random variables , 2004, math/0410168.

[14]  Arnaud Guillin,et al.  Transportation-information inequalities for Markov processes , 2007, 0706.4193.

[15]  David A. McAllester.  PAC-Bayesian model averaging , 1999, COLT '99.

[16]  S. Bobkov,et al.  Discrete isoperimetric and Poincaré-type inequalities , 1999 .

[17]  J. Dedecker,et al.  New dependence coefficients. Examples and applications to statistics , 2005 .

[18]  J. Dedecker,et al.  Parametrized Kantorovich-Rubinštein theorem and application to the coupling of random variables , 2006 .

[19]  Katalin Marton,et al.  A simple proof of the blowing-up lemma , 1986, IEEE Trans. Inf. Theory.

[20]  Shahar Mendelson,et al.  General nonexact oracle inequalities for classes with a subexponential envelope , 2012, 1206.0871.

[21]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[22]  Olivier Catoni,et al.  Statistical learning theory and stochastic optimization , 2004 .

[23]  K. Marton.  A measure concentration inequality for contracting markov chains , 1996 .

[24]  Jean-Yves Audibert,et al.  Robust linear least squares regression , 2010, 1010.0074.

[25]  S. Bobkov,et al.  Hypercontractivity of Hamilton-Jacobi equations , 2001 .

[26]  Journal Url,et al.  A tail inequality for suprema of unbounded empirical processes with applications to Markov chains , 2008 .

[27]  B. Maurey.  Some deviation inequalities , 1990, math/9201216.

[28]  Pierre Alquier,et al.  Model selection for weakly dependent time series forecasting , 2009, 0902.2924.

[29]  S. Bobkov,et al.  Poincaré’s inequalities and Talagrand’s concentration phenomenon for the exponential distribution , 1997 .

[30]  S. Boucheron,et al.  On concentration of self-bounding functions , 2009 .

[31]  Paul-Marie Samson,et al.  Displacement convexity of entropy and related inequalities on graphs , 2012, Probability Theory and Related Fields.

[32]  S. Bobkov,et al.  Exponential Integrability and Transportation Cost Related to Logarithmic Sobolev Inequalities , 1999 .

[33]  O. Wintenberger.  Deviation inequalities for sums of weakly dependent time series , 2009, 0911.1682.

[34]  M. Talagrand.  Concentration of measure and isoperimetric inequalities in product spaces , 1994, math/9406212.

[35]  P. Lezaud.  Chernoff-type bound for finite Markov chains , 1998 .

[36]  Pierre Alquier,et al.  Prediction of time series by statistical learning: general losses and fast rates , 2012, 1211.1847.

[37]  C. Villani,et al.  Weighted Csiszár-Kullback-Pinsker inequalities and applications to transportation inequalities , 2005 .

[38]  A. Guillin,et al.  Transportation cost-information inequalities and applications to random dynamical systems and diffusions , 2004, math/0410172.

[39]  Gábor Lugosi,et al.  Concentration Inequalities - A Nonasymptotic Theory of Independence , 2013, Concentration Inequalities.

[40]  Paul-Marie Samson,et al.  Concentration of measure inequalities for Markov chains and $\Phi$-mixing processes , 2000 .

[41]  O. Wintenberger,et al.  ASYMPTOTIC NORMALITY OF THE QUASI MAXIMUM LIKELIHOOD ESTIMATOR FOR MULTIDIMENSIONAL CAUSAL PROCESSES , 2007, 0712.0679.

[42]  Sharp Bounds for the Tails of Functionals of Markov Chains , 2010 .

[43]  N. Gozlan.  Transport-entropy inequalities on the line , 2012, 1203.0326.

[44]  G. Schechtman,et al.  Remarks on Talagrand’s deviation inequality for Rademacher functions , 1990, math/9201208.

[45]  E. Rio.  Inégalités de Hoeffding pour les fonctions lipschitziennes de suites dépendantes , 2000 .

[46]  S. Goldstein.  Maximal coupling , 1979 .

[47]  Christian L'eonard,et al.  Transport Inequalities. A Survey , 2010, 1003.3852.

[48]  Bernstein type’s concentration inequalities for symmetric Markov processes@@@Bernstein types concentration inequalities for symmetric Markov processes , 2013 .

[49]  K. Marton.  Bounding $\bar{d}$-distance by informational divergence: a method to prove measure concentration , 1996 .