Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension

We extend Valiant's learnability model to learning classes of concepts defined by regions in Euclidean space E". Our methods lead to a unified treatment of some of Valiant's results, along with previous results of Pearl and Devroye and Wagner on distribution-free convergence of certain pattern recognition algorithms. We show that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned. Using this parameter, we analyze the complexity and closure properties of learnable classes. Authors A. Blumer and D. Haussler gratefully acknowledge the support of NSF grant IST-8317918, author A. Ehrenfeucht the support of NSF grant MCS-8305245, and author M. Warmuth the support of the Faculty Research Committee of the University of California at Santa Cruz. Part of this work was done while A. Blumer was visiting the University of California at Santa Cruz and M. Warmuth the Univer-

[1]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[2]  Tzay Y. Young,et al.  Classification, Estimation and Pattern Recognition , 1974 .

[3]  David S. Johnson,et al.  Approximation algorithms for combinatorial problems , 1973, STOC.

[4]  Luc Devroye,et al.  A distribution-free performance bound in error estimation (Corresp.) , 1976, IEEE Trans. Inf. Theory.

[5]  R. Dudley.  Central Limit Theorems for Empirical Measures , 1978 .

[6]  Judea Pearl,et al.  ON THE CONNECTION BETWEEN THE COMPLEXITY AND CREDIBILITY OF INFERRED MODELS , 1978 .

[7]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[8]  David S. Johnson,et al.  Computers and In stractability: A Guide to the Theory of NP-Completeness. W. H Freeman, San Fran , 1979 .

[9]  Judea Pearl,et al.  Capacity and Error Estimates for Boolean Classifiers with Limited Complexity , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Temple F. Smith.  Occam's razor , 1980, Nature.

[11]  Richard M. Dudley,et al.  Some special vapnik-chervonenkis classes , 1981, Discret. Math..

[12]  P. Assouad.  Densité et dimension , 1983 .

[13]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[14]  D. T. Lee,et al.  Computational Geometry—A Survey , 1984, IEEE Transactions on Computers.

[15]  Nimrod Megiddo,et al.  Linear Programming in Linear Time When the Dimension Is Fixed , 1984, JACM.

[16]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[17]  Franco P. Preparata,et al.  Correction to "Computational Geometry - A Survey" , 1985 .

[18]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[19]  Silvio Micali,et al.  How to construct random functions , 1986, JACM.

[20]  David Haussler,et al.  Occam's Razor , 1987, Inf. Process. Lett..