Learning from Examples with Unspeciied Attribute Values

We introduce the UAV learning model in which some of the attributes in the examples are unspeciied. In our model, an example x is classiied positive (resp., negative) if all possible assignments for the unspeciied attributes result in a positive (resp., negative) classiication. Otherwise the classiication given to x is \?" (for unknown). Given an example x in which some attributes are unspeciied, the oracle UAV-MQ responds with the classiication of x. Given a hypothesis h, the oracle UAV-EQ returns 1 an example x (that could have unspeciied attributes) for which h(x) is incorrect. We show that any class learnable in the exact model using the MQ and EQ oracles is also learnable in the UAV model using the MQ and UAV-EQ oracles as long as the counterexamples provided by the UAV-EQ oracle have a logarithmic number of unspeciied attributes. We also show that any class learnable in the exact model using the MQ and EQ oracles is also learnable in the UAV model using the UAV-MQ and UAV-EQ oracles as well as an oracle to evaluate a given boolean formula on an example with unspeciied attributes. (For some hypothesis classes such as decision trees and unate formulas the evaluation can be done in polynomial time without an oracle.) We also study the learnability of a universal class of decision trees under the UAV model and of DNF formulas under a representation-dependent variation of the UAV model.

[1]  Ray Bareiss,et al.  Concept Learning and Heuristic Classification in WeakTtheory Domains , 1990, Artif. Intell..

[2]  Shai Ben-David,et al.  Learning with restricted focus of attention , 1993, COLT '93.

[3]  Marek Karpinski,et al.  Learning read-once formulas using membership queries , 1989, COLT '89.

[4]  Manuel Blum,et al.  Equivalence of Free Boolean Graphs can be Decided Probabilistically in Polynomial Time , 1980, Inf. Process. Lett..

[5]  D. Angluin.  Queries and Concept Learning , 1988 .

[6]  György Turán,et al.  Learning with queries but incomplete information (extended abstract) , 1994, COLT '94.

[7]  Michael Frazier,et al.  Learning from a Consistently Ignorant Teacher , 1996, J. Comput. Syst. Sci..

[8]  Sampath Kannan,et al.  Oracles and Queries That Are Sufficient for Exact Learning , 1996, J. Comput. Syst. Sci..

[9]  George Shackelford,et al.  Learning k-DNF with noise in the attributes , 1988, Annual Conference Computational Learning Theory.

[10]  Harry B. Hunt,et al.  Monotone Boolean Formulas, Distributive Lattices, and the Complexities of Logics, Algebraic Structures, and Computation Structures (Preliminary Report) , 1986, STACS.

[11]  Michael Frazier,et al.  Learning from a consistently ignorant teacher , 1994, COLT '94.

[12]  Adam J. GroveNEC.  Learning Active Classiiers , 1996 .

[13]  D. Angluin,et al.  Randomly fallible teachers: Learning monotone DNF with an incomplete membership oracle , 1991, Machine Learning.

[14]  Nick Littlestone,et al.  Redundant noisy attributes, attribute errors, and linear-threshold learning using winnow , 1991, COLT '91.

[15]  Erik Meineche Schmidt,et al.  The Complexity of Equivalence and Containment for Free Single Variable Program Schemes , 1978, ICALP.

[16]  Donna K. Slonim,et al.  Learning with unreliable boundary queries , 1995, COLT '95.

[17]  Yasubumi Sakakibara,et al.  On Learning from Queries and Counterexamples in the Presence of Noise , 1991, Inf. Process. Lett..

[18]  Hans Ulrich Simon,et al.  On Restricted-Focus-of-Attention Learnability of Boolean Functions , 1998, Machine Learning.

[19]  Ming Li,et al.  Learning in the presence of malicious errors , 1993, STOC '88.

[20]  Nader H. Bshouty,et al.  Exact learning via the Monotone theory , 1993, Proceedings of 1993 IEEE 34th Annual Foundations of Computer Science.

[21]  Dale Schuurmans,et al.  Learning to classify incomplete examples , 1997, COLT 1997.

[22]  Vijay Raghavan,et al.  Read-Twice DNF Formulas are Properly Learnable , 1994, Inf. Comput..

[23]  H. Aizenstein,et al.  Exact learning of read-twice DNF formulas , 1991, [1991] Proceedings 32nd Annual Symposium of Foundations of Computer Science.

[24]  P. Laird.  Learning from Good and Bad Data , 1988 .

[25]  Stephen Kwek,et al.  Learning from examples with unspecified attribute values (extended abstract) , 1997, COLT '97.

[26]  Robert E. Schapire,et al.  Efficient distribution-free learning of probabilistic concepts , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[27]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[28]  Hans Ulrich Simon,et al.  Structural Results about Exact Learning with Unspecified Attribute Values , 2000, J. Comput. Syst. Sci..

[29]  Lisa Hellerstein,et al.  Learning Arithmetic Read-Once Formulas , 1995, SIAM J. Comput..

[30]  J. Ross Quinlan,et al.  Unknown Attribute Values in Induction , 1989, ML.

[31]  Robert E. Schapire,et al.  Efficient Distribution-Free Learning of Probabilistic , 1994 .

[32]  Robert H. Sloan,et al.  Four Types of Noise in Data for PAC Learning , 1995, Inf. Process. Lett..

[33]  Avrim Blum,et al.  Fast learning of k-term DNF formulas with queries , 1992, STOC '92.

[34]  Nader H. Bshouty.  Simple learning algorithms using divide and conquer , 1995, COLT '95.

[35]  Dana Angluin,et al.  Learning with malicious membership queries and exceptions (extended abstract) , 1994, COLT '94.

[36]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[37]  Marek Karpinski,et al.  Learning read-once formulas with queries , 1993, JACM.

[38]  Dale Schuurmans,et al.  Learning Default Concepts , 1994 .