Learning by distances

A model of learning by distances is presented. In this model a concept is a point in a metric space. At each step of the learning process the student guesses a hypothesis and receives from the teacher an approximation of its distance to the target. A notion of a distance measuring the proximity of a hypothesis to the correct answer is common to many models of learnability. By focusing on this fundamental aspect we discover some general and simple tools for the analysis of learnability tasks. As a corollary we present new learning algorithms for Valiant?s PAC scenario with any given distribution. These algorithms can learn any PAC-learnable class and, in some cases, settle for significantly less information than the usual labeled examples. Insight gained by the new model is applied to show that every class of subsets C that has a finite VC-dimension is PAC-learnable with respect to any fixed distribution. Previously known results of this nature were subject to complicated measurability constraints.

[1]  Dana Angluin.  Equivalence queries and approximate fingerprints , 1989, COLT '89.

[2]  Yishay Mansour,et al.  A parametrization scheme for classifying models of learnability , 1989, COLT '89.

[3]  David Haussler,et al.  Generalizing the PAC model: sample size bounds from metric dimension-based uniform convergence results , 1989, 30th Annual Symposium on Foundations of Computer Science.

[4]  Noam Nisan,et al.  Constant depth circuits, Fourier transform, and learnability , 1989, 30th Annual Symposium on Foundations of Computer Science.

[5]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[6]  Tao Jiang,et al.  Learning regular languages from counterexamples , 1988, COLT '88.

[7]  Alon Itai,et al.  Learnability by fixed distributions , 1988, COLT '88.

[8]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[9]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[10]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[11]  R. Dudley.  A course on empirical processes , 1984 .

[12]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[13]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[14]  W. Hoeffding.  Probability inequalities for sum of bounded random variables , 1963 .

[15]  J. W. Alexander.  On the Class of a Covariant Tensor , 1926 .

[16]  Dana Angluin.  Negative results for equivalence queries , 1990, Mach. Learn..

[17]  Dana Angluin.  Equivalence queries and approximate fingerprints , 1989, COLT '89.

[18]  Alon Itai,et al.  Learnability by fixed distributions , 1988, COLT '88.

[19]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[20]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[21]  R. Dudley.  A course on empirical processes , 1984 .