Approximating Hyper-Rectangles: Learning and Pseudorandom Sets

The PAC learning of rectangles has been studied because they have been found experimentally to yield excellent hypotheses for several applied learning problems. Also, pseudorandom sets for rectangles have been actively studied recently because (i) they are a subproblem common to the derandomization of depth-2 (DNF) circuits and derandomizing randomized logspace, and (ii) they approximate the distribution ofnindependent multivalued random variables. We present improved upper bounds for a class of such problems of “approximating” high-dimensional rectangles that arise in PAC learning and pseudo- randomness.

[1]  Noga Alon,et al.  Explicit Ramsey graphs and orthonormal labelings , 1994, Electron. J. Comb..

[2]  Noga Alon,et al.  Simple Construction of Almost k-wise Independent Random Variables , 1992, Random Struct. Algorithms.

[3]  Noga Alon,et al.  A Fast and Simple Randomized Parallel Algorithm for the Maximal Independent Set Problem , 1985, J. Algorithms.

[4]  Noam Nisan,et al.  Pseudorandomness for network algorithms , 1994, STOC '94.

[5]  Noam Nisan Extracting randomness: how and why , 1996 .

[6]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[7]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[8]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[9]  Michael E. Saks,et al.  Discrepancy sets and pseudorandom generators for combinatorial rectangles , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[10]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[11]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[12]  Larry Carter,et al.  Universal Classes of Hash Functions , 1979, J. Comput. Syst. Sci..

[13]  Leonard Pitt,et al.  Prediction-Preserving Reducibility , 1990, J. Comput. Syst. Sci..

[14]  Michael Luby A Simple Parallel Algorithm for the Maximal Independent Set Problem , 1986, SIAM J. Comput..

[15]  P. Erdös Some remarks on the theory of graphs , 1947 .

[16]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[17]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[18]  Peter Frankl,et al.  Intersection theorems with geometric consequences , 1981, Comb..

[19]  Peter Auer,et al.  On Learning From Multi-Instance Examples: Empirical Evaluation of a Theoretical Approach , 1997, ICML.

[20]  Noam Nisan,et al.  Approximations of general independent distributions , 1992, STOC '92.

[21]  Michael Krivelevich,et al.  Bounding Ramsey Numbers through Large Deviation Inequalities , 1995, Random Struct. Algorithms.

[22]  A. Selman Structure in Complexity Theory , 1986, Lecture Notes in Computer Science.

[23]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[24]  H. L. Abbott A Note on Ramsey's Theorem , 1972, Canadian Mathematical Bulletin.

[25]  Moni Naor,et al.  Small-Bias Probability Spaces: Efficient Constructions and Applications , 1993, SIAM J. Comput..

[26]  Philip M. Long,et al.  PAC Learning Axis-Aligned Rectangles with Respect to Product Distributions from Multiple-Instance Examples , 1996, COLT.

[27]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[28]  Joel H. Spencer,et al.  Asymptotic lower bounds for Ramsey functions , 1977, Discret. Math..

[29]  D. Pollard Convergence of stochastic processes , 1984 .

[30]  Michael E. Saks,et al.  Efficient construction of a small hitting set for combinatorial rectangles in high dimension , 1993, Comb..

[31]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[32]  David Haussler,et al.  Predicting {0,1}-functions on randomly drawn points , 1988, COLT '88.

[33]  Sholom M. Weiss,et al.  Computer Systems That Learn , 1990 .