A composition theorem for learning algorithms with applications to geometric concept classes

This paper solves the open problem of exact learning geometric objects bounded by hyperplanes (and more generally by any constant degree algebraic surfaces) in the constant dimensional space from equivalence queries only (i.e., in the on-line learning model). We present a novel approach that allows, under certain conditions, the composition of learning algorithms for simple classes into an algorithm for a more complicated class. Informally speaking, it shows that if a class of concepts C is learnable in time t using a small space then @, the class of all functions of the form ~(gl, . . , gm) With 91, . . . . gm E C and any boolean function ~, is learnable in polynomial time in t and m. We then show that the class of halfspaces in a fixed dimension space is learnable with a small space.

[1]  Paul W. Goldberg,et al.  Bounding the Vapnik-Chervonenkis Dimension of Concept Classes Parameterized by Real Numbers , 1993, COLT '93.

[2]  Foued Ameur A space-bounded learning algorithm for axis-parallel rectangles , 1995, EuroCOLT.

[3]  Friedhelm Meyer auf der Heide,et al.  Trial and error: a new approach to space-bounded learning , 1994, EuroCOLT.

[4]  Zhixiang Chen,et al.  On learning discretized geometric concepts , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[5]  Hans Ulrich Simon,et al.  Learning decision lists and trees with equivalence-queries , 1995, EuroCOLT.

[6]  Saburo Muroga,et al.  Threshold logic and its applications , 1971 .

[7]  Norbert Sauer,et al.  On the Density of Families of Sets , 1972, J. Comb. Theory, Ser. A.

[8]  Sally Floyd,et al.  Space-bounded learning and the Vapnik-Chervonenkis dimension , 1989, COLT '89.

[9]  H. Warren Lower bounds for approximation by nonlinear manifolds , 1968 .

[10]  Manfred K. Warmuth,et al.  Learning Nested Differences of Intersection-Closed Concept Classes , 1989, COLT '89.

[11]  N. Littlestone Mistake bounds and logarithmic linear-threshold learning algorithms , 1990 .

[12]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[13]  Wolfgang Maass,et al.  On the complexity of learning from counterexamples , 1989, 30th Annual Symposium on Foundations of Computer Science.

[14]  Peter Auer,et al.  On-line learning of rectangles in noisy environments , 1993, COLT '93.

[15]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[16]  Subhash Suri,et al.  Noise-tolerant distribution-free learning of general geometric concepts , 1996, STOC '96.

[17]  Zhixiang Chen,et al.  On-line learning of rectangles , 1992, COLT '92.