The query complexity of finding local minima in the lattice

In this paper we study the query complexity of finding local minimum points of a boolean function. This task occurs frequently in exact learning algorithms for many natural classes, such as monotone DNF, 0( log n) -term DNF, unate DNF and decision trees. On the negative side, we prove that any (possibly randomized) algorithm that produces a local minimum of a function f chosen from a sufficiently “rich” concept class, using a membership oracle for f, must ask fl(n”) membership queries in the worst case. In particular, this lower bound applies to the class of decision trees. A simple algorithm is known that achieves this lower bound. On the positive side, we show that for the class O(log n)-term DNF finding local minimum points requires only O(n log n) membership queries (and more generally O( nt) membership queries for tterm DNF with t < n). This efficient procedure improves the time and query complexity of known learning algorithms for the class 0( log n)-term DNF.

[1]  Ronald L. Rivest,et al.  Inference of finite automata using homing sequences , 1989, STOC '89.

[2]  Linda Sellie,et al.  Learning sparse multivariate polynomials over a field with queries and counterexamples , 1993, COLT '93.

[3]  Andrew C. Yao,et al.  Lower bounds by probabilistic arguments , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).

[4]  Nader H. Bshouty,et al.  Asking questions to minimize errors , 1993, COLT '93.

[5]  Donna K. Slonim,et al.  Learning with unreliable boundary queries , 1995, COLT '95.

[6]  Wolfgang Maass,et al.  On the complexity of learning from counterexamples and membership queries , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[7]  Eyal Kushilevitz,et al.  On the applications of multiplicity automata in learning , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[8]  A. Rosser.  A.I.D.S. , 1986, Maryland medical journal.

[9]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[10]  Ron M. Roth,et al.  Interpolation and Approximation of Sparse Multivariate Polynomials over GF(2) , 1991, SIAM J. Comput..

[11]  Francesco Bergadano,et al.  Learning Sat-k-DNF formulas from membership queries , 1996, STOC '96.

[12]  Eyal Kushilevitz,et al.  A Simple Algorithm for Learning O (log n)-Term DNF , 1997, Inf. Process. Lett..

[13]  Nader H. Bshouty.  Simple learning algorithms using divide and conquer , 1995, COLT '95.

[14]  Nader H. Bshouty.  Exact Learning Boolean Function via the Monotone Theory , 1995, Inf. Comput..

[15]  Eyal Kushilevitz.  A simple algorithm for learning O(log n)-term DNF , 1996, COLT '96.

[16]  Marek Karpinski,et al.  Learning read-once formulas with queries , 1993, JACM.

[17]  Tibor Hegedüs,et al.  Generalized Teaching Dimensions and the Query Complexity of Learning , 1995, COLT.

[18]  Aravind Srinivasan,et al.  Splitters and near-optimal derandomization , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[19]  Sally A. Goldman,et al.  Learning k-term DNF formulas with an incomplete membership oracle , 1992, COLT '92.

[20]  Avrim Blum,et al.  Fast learning of k-term DNF formulas with queries , 1992, STOC '92.

[21]  Marek Karpinski,et al.  On Zero-Testing and Interpolation of k-Sparse Multivariate Polynomials Over Finite Fields , 1991, Theor. Comput. Sci..

[22]  Lisa Hellerstein,et al.  How many queries are needed to learn? , 1995, JACM.

[23]  D. Angluin,et al.  Randomly fallible teachers: Learning monotone DNF with an incomplete membership oracle , 1991, Machine Learning.

[24]  Tibor Hegedűs,et al.  Generalized teaching dimensions and the query complexity of learning , 1995, Annual Conference Computational Learning Theory.

[25]  Wolfgang Maass,et al.  On the complexity of learning from counterexamples , 1989, 30th Annual Symposium on Foundations of Computer Science.

[26]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..