Randomly fallible teachers: Learning monotone DNF with an incomplete membership oracle

We introduce a new fault-tolerant model of algorithmic learning using an equivalence oracle and an incomplete membership oracle, in which the answers to a random subset of the learner's membership queries may be missing. We demonstrate that, with high probability, it is still possible to learn monotone DNF formulas in polynomial time, provided that the fraction of missing answers is bounded by some constant less than one. Even when half the membership queries are expected to yield no information, our algorithm will exactly identify m-term, n-variable monotone DNF formulas with an expected O(mn2) queries. The same task has been shown to require exponential time using equivalence queries alone. We extend the algorithm to handle some one-sided errors, and discuss several other possible error models. It is hoped that this work may lead to a better understanding of the power of membership queries and the effects of faulty teachers on query models of concept learning.

[1]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[2]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[3]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[4]  Leslie G. Valiant,et al.  On the learnability of Boolean formulae , 1987, STOC.

[5]  Philip D. Laird,et al.  Learning from good data and bad , 1987 .

[6]  N. Littlestone.  Learning Quickly When Irrelevant Attributes Abound: A New Linear-Threshold Algorithm , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[7]  Ming Li,et al.  Learning in the presence of malicious errors , 1993, STOC '88.

[8]  George Shackelford,et al.  Learning k-DNF with noise in the attributes , 1988, Annual Conference Computational Learning Theory.

[9]  M. Kearns,et al.  Crytographic limitations on learning Boolean formulae and finite automata , 1989, STOC '89.

[10]  Robert H. Sloan,et al.  Computational learning theory: new models and algorithms , 1989 .

[11]  R. Schapire,et al.  Exact identification of circuits using fixed points of amplification functions , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[12]  Robert E. Schapire,et al.  Exact identification of circuits using fixed points of amplification functions , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[13]  Stephen R. Schach,et al.  Learning switch configurations , 1990, COLT '90.

[14]  Dana Angluin.  Negative results for equivalence queries , 1990, Mach. Learn..

[15]  Michael Kearns,et al.  Computational complexity of machine learning , 1990, ACM distinguished dissertations.

[16]  Thomas R. Hancock,et al.  Identifying μ-formula decision trees with queries , 1990, COLT '90.

[17]  Dana Angluin,et al.  When won't membership queries help? , 1991, STOC '91.

[18]  Yasubumi Sakakibara,et al.  On Learning from Queries and Counterexamples in the Presence of Noise , 1991, Inf. Process. Lett..

[19]  Donna K. Slonim,et al.  Learning Monotone DNF with an Incomplete Membership Oracle , 1991, COLT.

[20]  Wolfgang Maass,et al.  On-line learning with an oblivious environment and the power of randomization , 1991, COLT '91.

[21]  Robert H. Sloan,et al.  Corrigendum to types of noise in data for concept learning , 1988, COLT '92.

[22]  Marek Karpinski,et al.  Learning read-once formulas with queries , 1993, JACM.

[23]  Leslie G. Valiant,et al.  Cryptographic Limitations on Learning Boolean Formulae and Finite Automata , 1993, Machine Learning: From Theory to Applications.