When are epsilon-nets small?

In many interesting situations the size of epsilon-nets depends only on $\epsilon$ together with different complexity measures. The aim of this paper is to give a systematic treatment of such complexity measures arising in Discrete and Computational Geometry and Statistical Learning, and to bridge the gap between the results appearing in these two fields. As a byproduct, we obtain several new upper bounds on the sizes of epsilon-nets that generalize/improve the best known general guarantees. In particular, our results work with regimes when small epsilon-nets of size $o(\frac{1}{\epsilon})$ exist, which are not usually covered by standard upper bounds. Inspired by results in Statistical Learning we also give a short proof of the Haussler's upper bound on packing numbers.

[1]  K. Alexander,et al.  Rates of growth and sample moduli for weighted empirical processes indexed by sets , 1987 .

[2]  Nabil H. Mustafa,et al.  Epsilon-Approximations & Epsilon-Nets , 2017, 1702.03676.

[3]  Nabil H. Mustafa.  A Simple Proof of the Shallow Packing Lemma , 2016, Discrete & Computational Geometry.

[4]  W. Lockau,et al.  Contents , 2015 .

[5]  Yi Li,et al.  Using the doubling dimension to analyze the generalization of learning algorithms , 2009, J. Comput. Syst. Sci..

[6]  David Haussler,et al.  Sphere Packing Numbers for Subsets of the Boolean n-Cube with Bounded Vapnik-Chervonenkis Dimension , 1995, J. Comb. Theory, Ser. A.

[7]  S. Boucheron,et al.  Theory of classification : a survey of some recent advances , 2005 .

[8]  Steve Hanneke,et al.  Theory of Disagreement-Based Active Learning , 2014, Found. Trends Mach. Learn..

[9]  P. Bartlett,et al.  Local Rademacher complexities , 2005, math/0508275.

[10]  Shun-ichi Amari,et al.  A Theory of Pattern Recognition , 1968 .

[11]  Nabil H. Mustafa,et al.  A Simple Proof of Optimal Epsilon Nets , 2018, Comb..

[12]  Liu Yang,et al.  Minimax Analysis of Active Learning , 2014, J. Mach. Learn. Res..

[13]  Micha Sharir,et al.  Small-Size $\eps$-Nets for Axis-Parallel Rectangles and Boxes , 2010, SIAM J. Comput..

[14]  L. Lecam.  Convergence of Estimates Under Dimensionality Restrictions , 1973 .

[15]  Kasturi R. Varadarajan.  Weighted geometric set cover via quasi-uniform sampling , 2010, STOC '10.

[16]  Arijit Ghosh,et al.  Two Proofs for Shallow Packings , 2016, Discret. Comput. Geom..

[17]  Timothy M. Chan,et al.  Weighted capacitated, priority, and geometric set cover via improved quasi-uniform sampling , 2012, SODA.

[18]  Nikita Zhivotovskiy.  Optimal learning via local entropies and sample compression , 2017, COLT.

[19]  Alon Itai,et al.  Learnability with Respect to Fixed Distributions , 1991, Theor. Comput. Sci..

[20]  Maria-Florina Balcan,et al.  Active and passive learning of linear separators under log-concave distributions , 2012, COLT.

[21]  Hans Ulrich Simon,et al.  Recursive teaching dimension, VC-dimension and sample compression , 2014, J. Mach. Learn. Res..

[22]  R. Dudley.  A course on empirical processes , 1984 .

[23]  Robert Krauthgamer,et al.  Bounded geometries, fractals, and low-distortion embeddings , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[24]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[25]  János Komlós,et al.  Almost tight bounds forɛ-Nets , 1992, Discret. Comput. Geom..

[26]  Gábor Lugosi,et al.  Introduction to Statistical Learning Theory , 2004, Advanced Lectures on Machine Learning.

[27]  David Haussler,et al.  Predicting {0,1}-functions on randomly drawn points , 1988, COLT '88.

[28]  David Haussler,et al.  ɛ-nets and simplex range queries , 1987, Discret. Comput. Geom..

[29]  H. Balsters,et al.  Learnability with respect to fixed distributions , 1991 .

[30]  Michael Kearns,et al.  On the complexity of teaching , 1991, COLT '91.

[31]  David Haussler,et al.  Epsilon-nets and simplex range queries , 1986, SCG '86.

[32]  V. Koltchinskii,et al.  Concentration inequalities and asymptotic results for ratio type empirical processes , 2006, math/0606788.

[33]  Steve Hanneke,et al.  Localization of VC Classes: Beyond Local Rademacher Complexities , 2016, ALT.

[34]  Kamalika Chaudhuri,et al.  Beyond Disagreement-Based Agnostic Active Learning , 2014, NIPS.

[35]  Shahar Mendelson,et al.  `local' vs. `global' parameters -- breaking the gaussian complexity barrier , 2015, 1504.02191.

[36]  Steve Hanneke,et al.  The Optimal Sample Complexity of PAC Learning , 2015, J. Mach. Learn. Res..