Malicious Omissions and Errors in Answers to Membership Queries

We consider two issues in polynomial-time exact learning of concepts using membership and equivalence queries: (1) errors or omissions in answers to membership queries, and (2) learning finite variants of concepts drawn from a learnable class.To study (1), we introduce two new kinds of membership queries: limited membership queries and malicious membership queries. Each is allowed to give incorrect responses on a maliciously chosen set of strings in the domain. Instead of answering correctly about a string, a limited membership query may give a special “I don't know” answer, while a malicious membership query may give the wrong answer. A new parameter Lis used to bound the length of an encoding of the set of strings that receive such incorrect answers. Equivalence queries are answered correctly, and learning algorithms are allowed time polynomial in the usual parameters and L. Any class of concepts learnable in polynomial time using equivalence and malicious membership queries is learnable in polynomial time using equivalence and limited membership queries; the converse is an open problem. For the classes of monotone monomials and monotone k-term DNF formulas, we present polynomial-time learning algorithms using limited membership queries alone. We present polynomial-time learning algorithms for the class of monotone DNF formulas using equivalence and limited membership queries, and using equivalence and malicious membership queries.To study (2), we consider classes of concepts that are polynomially closed under finite exceptions and a natural operation to add exception tables to a class of concepts. Applying this operation, we obtain the class of monotone DNF formulas with finite exceptions. We give a polynomial-time algorithm to learn the class of monotone DNF formulas with finite exceptions using equivalence and membership queries. We also give a general transformation showing that any class of concepts that is polynomially closed under finite exceptions and is learnable in polynomial time using standard membership and equivalence queries is also polynomial-time learnable using malicious membership and equivalence queries. Corollaries include the polynomial-time learnability of the following classes using malicious membership and equivalence queries: deterministic finite acceptors, boolean decision trees, and monotone DNF formulas with finite exceptions.

[1]  William J. Bultman.  Topics in the theory of machine learning and neural computing , 1991 .

[2]  D. Angluin.  Queries and Concept Learning , 1988 .

[3]  Robert E. Schapire,et al.  Exact Identification of Read-Once Formulas Using Fixed Points of Amplification Functions , 1993, SIAM J. Comput..

[4]  Eric B. Baum,et al.  Neural net algorithms that learn in polynomial time from examples and queries , 1991, IEEE Trans. Neural Networks.

[5]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[6]  Sally A. Goldman,et al.  Learning k-term DNF formulas with an incomplete membership oracle , 1992, COLT '92.

[7]  Philip M. Long,et al.  Simulating access to hidden information while learning , 1994, STOC '94.

[8]  Michael Frazier,et al.  Learning from a consistently ignorant teacher , 1994, COLT '94.

[9]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[10]  Leslie Pack Kaelbling,et al.  Inferring finite automata with stochastic output functions and an application to map learning , 1992, 26th Annual Symposium on Foundations of Computer Science (sfcs 1985).

[11]  D. Angluin,et al.  Randomly fallible teachers: Learning monotone DNF with an incomplete membership oracle , 1991, Machine Learning.

[12]  Wolfgang Maass,et al.  Lower bound methods and separation results for on-line learning models , 1992, Machine Learning.

[13]  Kenneth Basye,et al.  Inferring Finite Automata with Stochastic Output Functions and an Application to Map Learning , 2004, Machine Learning.

[14]  Yasubumi Sakakibara,et al.  On Learning from Queries and Counterexamples in the Presence of Noise , 1991, Inf. Process. Lett..

[15]  Wolfgang Maass,et al.  Lower Bound Methods and Separation Results for On-Line Learning Models , 2004, Machine Learning.

[16]  FunctionsSally,et al.  Exact Identi cation of Read-once Formulas Using Fixed Points ofAmpli cation , 1992 .

[17]  Dana Angluin,et al.  Learning with malicious membership queries and exceptions (extended abstract) , 1994, COLT '94.

[18]  György Turán,et al.  Learning with queries but incomplete information (extended abstract) , 1994, COLT '94.

[19]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[20]  Leonard Pitt,et al.  On the Necessity of Occam Algorithms , 1992, Theor. Comput. Sci..

[21]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[22]  Ronitt Rubinfeld,et al.  Learning fallible Deterministic Finite Automata , 1995, Machine Learning.

[23]  John R. Anderson.  Cognitive Psychology and Its Implications , 1980 .

[24]  Eyal Kushilevitz,et al.  Learning decision trees using the Fourier spectrum , 1991, STOC '91.

[25]  Yu. I. Zhuravlev,et al.  REALIZATION OF BOOLEAN FUNCTIONS WITH A SMALL NUMBER OF ZEROS BY DISJUNCTIVE NORMAL FORMS, AND RELATED PROBLEMS , 1985 .

[26]  Dana Angluin.  Exact learning of p-dnf formulas with malicious membership queries , 1993 .