Learnability in Valiant’s PAC learning model has been shown to be strongly related to the existence of uniform laws of large numbers. These laws define a distribution-free convergence property of means to expectations uniformly over classes of random variables. Classes of real-valued functions enjoying such a properly are also known as uniform Glivenko-Cantelli classes. In this paper we prove, through a generalization of Sauer’s lemma that may be interesting in its own righi, a new characterization of uniform Glivenko-Cantelli classes. Our characterization yields Dudley, Gint!, and Zinn’s previous characterization as a corollary. Furthermore, it is the first based on a simple combinatorial quantify generalizing the VapnikChervonenkis dimension. We apply this result to characterize PAC leamability in the statistical regression framework of probabilistic concepts, solving an open problem posed by Keams and Schapire. Our characterization shows that the accuracy parameter plays a crucial role in determining the effective complexity of the learner’s hypothesis class.
[1]
Vladimir Vapnik,et al.
Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities
,
1971
.
[2]
Norbert Sauer,et al.
On the Density of Families of Sets
,
1972,
J. Comb. Theory A.
[3]
V. Milman.
Some remarks about embeddings ofl1k in finite-dimensional spaces
,
1982
.
[4]
N. Alon,et al.
Embedding ofl∞k in finite dimensional Banach spaces
,
1983
.
[5]
D. Pollard.
Convergence of stochastic processes
,
1984
.
[6]
Peter W. Shor,et al.
A lower bound for 0, 1, * tournament codes
,
1987,
Discret. Math..
[7]
David Haussler,et al.
Occam's Razor
,
1987,
Inf. Process. Lett..
[8]
D. Pollard.
Empirical Processes: Theory and Applications
,
1990
.
[9]
David Haussler,et al.
Decision Theoretic Generalizations of the PAC Model for Neural Net and Other Learning Applications
,
1992,
Inf. Comput..
[10]
Philip M. Long,et al.
A Generalization of Sauer's Lemma
,
1995,
J. Comb. Theory A.