Adaptive versus nonadaptive attribute-efficient learning

We study the complexity of learning arbitrary Boolean functions of n variables by membership queries, if at most r variables are relevant. Problems of this type have important applications in fault searching, e.g. logical circuit testing and generalized group testing. Previous literature concentrates on special classes of such Boolean functions and considers only adaptive strategies. First we give a straightforward adaptive algorithm using O(r2r log n) queries, but actually, most queries are asked nonadaptively. This leads to the problem of purely nonadaptive learning. We give a graph-theoretic characterization of nonadaptive learning families, called r-wise bipartite connected families. By the probabilistic method we show the existence of such families of size O(r2r log n + r22r). This implies that nonadaptive attribute-efficient learning is not essentially more expensive than adaptive learning. We also sketch an explicit pseudopolynomial construction, though with a slightly worse bound. It uses the common derandomization technique of small-biased k-independent sample spaces. For the special case r = 2, we get roughly 2.275 log n adaptive queries, which is fairly close to the obvious lower bound of 2 log n. For the class of monotone functions, we prove that the optimal query number O(2r + r log n) can be already achieved in O(r) stages. On the other hand, Ω(2r log n) is a lower bound on nonadaptive queries.

[1]  D. Angluin Queries and Concept Learning , 1988 .

[2]  Peter Damaschke Computational Aspects of Parallel Attribute-Efficient Learning , 1998, ALT.

[3]  Emanuel Knill,et al.  Lower bounds for identifying subset members with subset queries , 1994, SODA '95.

[4]  Eberhard Triesch,et al.  Edge search in graphs and hypergraphs of bounded rank , 1993, Discret. Math..

[5]  Emanuel Knill,et al.  A Comparative Survey of Non-Adaptive Pooling Designs , 1996 .

[6]  Joel H. Spencer,et al.  Families of k-independent sets , 1973, Discret. Math..

[7]  N. Littlestone Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[8]  Rajeev Motwani,et al.  Randomized Algorithms , 1995, SIGA.

[9]  Noga Alon,et al.  Simple Construction of Almost k-wise Independent Random Variables , 1992, Random Struct. Algorithms.

[10]  Gadiel Seroussi,et al.  Vector sets for exhaustive testing of logic circuits , 1988, IEEE Trans. Inf. Theory.

[11]  Nader H. Bshouty Exact Learning Boolean Function via the Monotone Theory , 1995, Inf. Comput..

[12]  Peter Damaschke Randomized Group Testing for Mutually Obscuring Defectives , 1998, Inf. Process. Lett..

[13]  Noga Alon,et al.  Simple construction of almost k-wise independent random variables , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[14]  Ron M. Roth,et al.  Interpolation and Approximation of Sparse Multivariate Polynomials over GF(2) , 1991, SIAM J. Comput..

[15]  Dan Roth,et al.  Reasoning with Models , 1994, Artif. Intell..

[16]  Annalisa De Bonis,et al.  Improved Algorithms for Group Testing with Inhibitors , 1998, Inf. Process. Lett..

[17]  Marek Karpinski,et al.  On Zero-Testing and Interpolation of k-Sparse Multivariate Polynomials Over Finite Fields , 1991, Theor. Comput. Sci..

[18]  David C. Torney,et al.  Optimal Pooling Designs with Error Detection , 1994, J. Comb. Theory, Ser. A.

[19]  Lisa Hellerstein,et al.  PAC learning with irrelevant attributes , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[20]  Ryuhei Uehara,et al.  Optimal Attribute-Efficient Learning of Disjunction, Parity and Threshold Functions , 1997, EuroCOLT.

[21]  Nader H. Bshouty,et al.  On the exact learning of formulas in parallel , 1992, Proceedings., 33rd Annual Symposium on Foundations of Computer Science.

[22]  Heikki Mannila,et al.  Learning hierarchical rule sets , 1992, COLT '92.

[23]  Eberhard Triesch A Group Testing Problem for Hypergraphs of Bounded Rank , 1996, Discret. Appl. Math..

[24]  Moni Naor,et al.  Small-Bias Probability Spaces: Efficient Constructions and Applications , 1993, SIAM J. Comput..

[25]  Eyal Kushilevitz,et al.  The query complexity of finding local minima in the lattice , 1998, COLT' 98.

[26]  Peter Damaschke A Tight Upper Bound for Group Testing in Graphs , 1994, Discret. Appl. Math..

[27]  S. Muthukrishnan,et al.  Group testing problems in experimental molecular biology , 1995, math/9505211.

[28]  Noga Alon,et al.  Construction of asymptotically good low-rate error-correcting codes through pseudo-random graphs , 1992, IEEE Trans. Inf. Theory.

[29]  Avrim Blum,et al.  Learning boolean functions in an infinite attribute space , 1990, STOC '90.

[30]  A. Macula,et al.  Simplified searching for two defects , 1998 .

[31]  Thomas Hofmeister An Application of Codes to Attribute-Efficient Learning , 1999, EuroCOLT.

[32]  Annalisa De Bonis,et al.  Improved Algorithms for Chemical Threshold Testing Problems , 1998, COCOON.

[33]  Ingo Wegener,et al.  On the Cut-off Point for Combinatorial Group Testing , 1999, Discret. Appl. Math..

[34]  D. Du,et al.  Combinatorial Group Testing and Its Applications , 1993 .

[35]  Lisa Hellerstein,et al.  Attribute-efficient learning in query and mistake-bound models , 1996, COLT '96.

[36]  Peter Damaschke A Chip Search Problem on Binary Numbers , 1998, LATIN.

[37]  C. Colbourn,et al.  The CRC handbook of combinatorial designs , edited by Charles J. Colbourn and Jeffrey H. Dinitz. Pp. 784. $89.95. 1996. ISBN 0-8493-8948-8 (CRC). , 1997, The Mathematical Gazette.

[38]  Thomas G. Dietterich,et al.  Learning Boolean Concepts in the Presence of Many Irrelevant Features , 1994, Artif. Intell..

[39]  Peter Damaschke The Algorithmic Complexity of Chemical Threshold Testing , 1997, CIAC.

[40]  Aravind Srinivasan,et al.  Splitters and near-optimal derandomization , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[41]  Lisa Hellerstein,et al.  Learning in the presence of finitely or infinitely many irrelevant attributes , 1991, COLT '91.