Learning Boxes in High Dimension

We present exact learning algorithms that learn several classes of (discrete) boxes in {0,..., l−1}n. In particular we learn: (1) The class of unions of O(log n) boxes in time poly(n, log l) (solving an open problem of [15, 11]). (2) The class of unions of disjoint boxes in time poly(n, t,log l), where t is the number of boxes. (Previously this was known only in the case where all boxes are disjoint in one of the dimensions). In particular our algorithm learns the class of decision trees (over n variables that take values in {0,..., l−1}) with comparison nodes in time poly (n, t, log l), where t is the number of leaves (this was an open problem in [8] which was shown in [3] to be learnable in time poly(n, t, l)). (3) The class of unions of O(1)-degenerate boxes (that is, boxes that depend only on O(1) variables) in time poly(n, t, log l) (generalizing the learnability of O(1)-DNF and of boxes in O(1) dimensions). The algorithm for this class uses only equivalence queries and it can also be used to learn the class of unions of O(1) boxes (from equivalence queries only).

[1]  Jeffrey C. Jackson.  An Efficient Membership-Query Algorithm for Learning DNF with Respect to the Uniform Distribution , 1997, J. Comput. Syst. Sci..

[2]  Nader H. Bshouty.  Exact Learning Boolean Function via the Monotone Theory , 1995, Inf. Comput..

[3]  Paul W. Goldberg,et al.  Learning unions of boxes with membership and equivalence queries , 1994, COLT '94.

[4]  Shai Ben-David,et al.  A composition theorem for learning algorithms with applications to geometric concept classes , 1997, STOC '97.

[5]  Manfred K. Warmuth,et al.  Efficient Learning With Virtual Threshold Gates , 1995, Inf. Comput..

[6]  W. Maass,et al.  On the complexity of learning from counterexamples , 1989, 30th Annual Symposium on Foundations of Computer Science.

[7]  Avrim Blum,et al.  Fast learning of k-term DNF formulas with queries , 1992, STOC '92.

[8]  Zhixiang Chen,et al.  The Bounded Injury Priority Method and the Learnability of Unions of Rectangles , 1996, Ann. Pure Appl. Log..

[9]  Francesco Bergadano,et al.  Learning Sat-k-DNF formulas from membership queries , 1996, STOC '96.

[10]  Eyal Kushilevitz,et al.  A Simple Algorithm for Learning O (log n)-Term DNF , 1997, Inf. Process. Lett..

[11]  J. C. Jackson.  The harmonic sieve: a novel application of Fourier analysis to machine learning theory and practice , 1996 .

[12]  Paul W. Goldberg,et al.  Exact Learning of Discretized Geometric Concepts , 1998, SIAM J. Comput..

[13]  Philip M. Long,et al.  Composite geometric concepts and polynomial predictability , 1990, COLT '90.

[14]  J. C. Jackson.  Learning Functions Represented as Multiplicity Automata , 1997 .

[15]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[16]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[17]  Nader H. Bshouty.  Simple learning algorithms using divide and conquer , 1995, COLT '95.

[18]  Subhash Suri,et al.  Noise-tolerant distribution-free learning of general geometric concepts , 1996, STOC '96.

[19]  Zhixiang Chen,et al.  On-line learning of rectangles , 1992, COLT '92.

[20]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[21]  Zhixiang Chen,et al.  On learning discretized geometric concepts , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[22]  Jeffrey C. Jackson,et al.  An efficient membership-query algorithm for learning DNF with respect to the uniform distribution , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[23]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[24]  Peter Auer,et al.  On-line learning of rectangles in noisy environments , 1993, COLT '93.

[25]  W. Maass,et al.  Eecient Learning with Virtual Threshold Gates , 1997 .