Covering Numbers for Convex Functions

In this paper, we study the covering numbers of the space of convex and uniformly bounded functions in multidimension. We find optimal upper and lower bounds for the <formula formulatype="inline"> <tex Notation="TeX">$\epsilon $</tex></formula>-covering number of <formula formulatype="inline"><tex Notation="TeX">$ {\cal C}([a, b]^{d}, B)$</tex></formula>, in the <formula formulatype="inline"><tex Notation="TeX">$L_{p}$</tex></formula>-metric, <formula formulatype="inline"><tex Notation="TeX">$1 \leq p < \infty $</tex></formula>, in terms of the relevant constants, where <formula formulatype="inline"> <tex Notation="TeX">$d \geq 1$</tex></formula>, <formula formulatype="inline"><tex Notation="TeX">$a < b \in \BBR $</tex></formula>, <formula formulatype="inline"><tex Notation="TeX">$B > 0$</tex></formula>, and <formula formulatype="inline"><tex Notation="TeX">$ {\cal C}([a,b]^{d}, B)$</tex></formula> denotes the set of all convex functions on <formula formulatype="inline"><tex Notation="TeX">$[a, b]^{d}$</tex></formula> that are uniformly bounded by <formula formulatype="inline"> <tex Notation="TeX">$B$</tex></formula>. We summarize previously known results on covering numbers for convex functions and also provide alternate proofs of some known results. Our results have direct implications in the study of rates of convergence of empirical minimization procedures as well as optimal convergence rates in the numerous convexity constrained function estimation problems.

[1]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[2]  Jon A Wellner,et al.  NONPARAMETRIC ESTIMATION OF MULTIVARIATE CONVEX-TRANSFORMED DENSITIES. , 2009, Annals of statistics.

[3]  L. Duembgen,et al.  APPROXIMATION BY LOG-CONCAVE DISTRIBUTIONS, WITH APPLICATIONS TO REGRESSION , 2010, 1002.3448.

[4]  D. Dryanov,et al.  Kolmogorov Entropy for Classes of Convex Functions , 2009 .

[5]  S. Geer.  Applications of empirical process theory , 2000 .

[6]  P. Milanfar,et al.  Convergence of algorithms for reconstructing convex bodies and directional measures , 2006, math/0608011.

[7]  P. Massart,et al.  Rates of convergence for minimum contrast estimators , 1993 .

[8]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[9]  M. Birman,et al.  PIECEWISE-POLYNOMIAL APPROXIMATIONS OF FUNCTIONS OF THE CLASSES $ W_{p}^{\alpha}$ , 1967 .

[10]  R. Dudley,et al.  Uniform Central Limit Theorems: Notation Index , 2014 .

[11]  R. Dudley.  Metric Entropy of Some Classes of Sets with Differentiable Boundaries , 1974 .

[12]  A. Kolmogorov,et al.  Entropy and "-capacity of sets in func-tional spaces , 1961 .

[13]  E. Bronshtein.  ε-Entropy of convex sets and functions , 1976 .

[14]  Aditya Guntuboyina.  Lower Bounds for the Minimax Risk Using $f$-Divergences, and Applications , 2011, IEEE Transactions on Information Theory.

[15]  L. Lecam.  Convergence of Estimates Under Dimensionality Restrictions , 1973 .

[16]  Lucien Birgé.  Approximation dans les espaces métriques et théorie de l'estimation , 1983 .

[17]  E. Seijo,et al.  Nonparametric Least Squares Estimation of a Multivariate Convex Regression Function , 2010, 1003.4765.

[18]  Julio M. Singer,et al.  Central Limit Theorems , 2011, International Encyclopedia of Statistical Science.

[19]  A. W. van der Vaart,et al.  Uniform Central Limit Theorems , 2001 .

[20]  D. Dunson,et al.  Bayesian nonparametric multivariate convex regression , 2011, 1109.0322.