We define embeddings between concept classes that are meant to reflect certain aspects of their combinatorial structure. Furthermore, we introduce a notion of uniwrsul concept chssrs ~ classes into which any member of a given family of classes can be embedded. These universal classes play a role similar to that played in computational complexity by languages that are hard for a given complexity class. We show that classes of half-spaces in [w” are universal with respect to families of algebraically defined classes. We present some combinatorial parameters along which the family of classes of a given VC-dimension can be grouped into sub-families. We use these parameters to investigate the existence of embeddings and the scope of universality of classes. We view the formulation of these parameters and the related questions that they raise as a significant component in this work. A second theme in our work is the notion of sample compression schemes. Intuitively, a class C has a sample compression scheme if for any finite sample, labeled according to a member of C, there exists a short sub-sample so that the labels of the full sample can be reconstructed from this sub-sample. By demonstrating the existence of certain compression schemes for the classes of half-spaces the existence of similar compression schemes for every class embeddable in half-spaces readily follows. We apply this approach to prove the existence of compression schemes for all ‘geometric concept classes’.
[1]
Vojtech Rödl,et al.
Geometrical realization of set systems and probabilistic communication complexity
,
1985,
26th Annual Symposium on Foundations of Computer Science (sfcs 1985).
[2]
Manfred K. Warmuth,et al.
Sample compression, learnability, and the Vapnik-Chervonenkis dimension
,
1995,
Machine Learning.
[3]
B. Natarajan.
On learning sets and functions
,
2004,
Machine Learning.
[4]
David Haussler,et al.
Learnability and the Vapnik-Chervonenkis dimension
,
1989,
JACM.
[5]
N. Littlestone.
Mistake bounds and logarithmic linear-threshold learning algorithms
,
1990
.
[6]
Shai Ben-David,et al.
A composition theorem for learning algorithms with applications to geometric concept classes
,
1997,
STOC '97.
[7]
Norbert Sauer,et al.
On the Density of Families of Sets
,
1972,
J. Comb. Theory, Ser. A.
[8]
M. Jerrum,et al.
Bounding the Vapnik-Chervonenkis Dimension of Concept Classes Parameterized by Real Numbers
,
2004,
Machine Learning.