The Query Complexity of Finding Local Minima in the Lattice

In this paper we study the query complexity of finding local minimum points of a boolean function. This task occurs frequently in exact learning algorithms for many natural classes, such as monotone DNF, O (log n)-term DNF, unate DNF, and decision trees. On the negative side, we prove that any (possibly randomized) algorithm that produces a local minimum of a function f chosen from a sufficiently "rich" concept class, using a membership oracle for f, must ask (n2) membership queries in the worst case. In particular, this lower bound applies to the class of decision trees. A simple algorithm is known that achieves this lower bound. On the positive side, we show that for the class O (log n)-term DNF finding local minimum points requires only (n log n) membership queries (and more generally (tn) membership queries for t-term DNF with tn). This efficient procedure improves the time and query complexity of known learning algorithms for the class O (log n)-term DNF. 2001 Elsevier Science.

[1]  Sally A. Goldman,et al.  Learning k-term DNF formulas with an incomplete membership oracle , 1992, COLT '92.

[2]  Wolfgang Maass,et al.  On the complexity of learning from counterexamples and membership queries , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[3]  Donna K. Slonim,et al.  Learning with unreliable boundary queries , 1995, COLT '95.

[4]  Lisa Hellerstein,et al.  How Many Queries Are Needed to Learn? , 1996, J. ACM.

[5]  Ronald L. Rivest,et al.  Inference of finite automata using homing sequences , 1989, STOC '89.

[6]  Francesco Bergadano,et al.  Learning Sat-k-DNF formulas from membership queries , 1996, STOC '96.

[7]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[8]  Eyal Kushilevitz,et al.  A Simple Algorithm for Learning O (log n)-Term DNF , 1997, Inf. Process. Lett..

[9]  Nader H. Bshouty,et al.  Asking questions to minimize errors , 1993, COLT '93.

[10]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[11]  Aravind Srinivasan,et al.  Splitters and near-optimal derandomization , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[12]  D. Angluin,et al.  Randomly fallible teachers: Learning monotone DNF with an incomplete membership oracle , 1991, Machine Learning.

[13]  Avrim Blum,et al.  Fast learning of k-term DNF formulas with queries , 1992, STOC '92.

[14]  Marek Karpinski,et al.  On Zero-Testing and Interpolation of k-Sparse Multivariate Polynomials Over Finite Fields , 1991, Theor. Comput. Sci..

[15]  Nader H. Bshouty,et al.  On the exact learning of formulas in parallel , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[16]  Eyal Kushilevitz,et al.  On the applications of multiplicity automata in learning , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[17]  D. Angluin.  Queries and Concept Learning , 1988 .

[18]  Linda Sellie,et al.  Learning sparse multivariate polynomials over a field with queries and counterexamples , 1993, COLT '93.

[19]  Ron M. Roth,et al.  Interpolation and Approximation of Sparse Multivariate Polynomials over GF(2) , 1991, SIAM J. Comput..

[20]  Marek Karpinski,et al.  Learning read-once formulas with queries , 1993, JACM.

[21]  Tibor Hegedűs,et al.  Generalized teaching dimensions and the query complexity of learning , 1995, Annual Conference Computational Learning Theory.

[22]  Wolfgang Maass,et al.  On the complexity of learning from counterexamples , 1989, 30th Annual Symposium on Foundations of Computer Science.

[23]  Andrew C. Yao,et al.  Lower bounds by probabilistic arguments , 1983, 24th Annual Symposium on Foundations of Computer Science (sfcs 1983).