Learning Conjunctive Concepts in Structural Domains

We study the problem of learning conjunctive concepts from examples on structural domains like the blocks world. This class of concepts is formally defined, and it is shown that even for samples in which each example (positive or negative) is a two-object scene, it is NP-complete to determine if there is any concept in this class that is consistent with the sample. We demonstrate how this result affects the feasibility of Mitchell's version of space approach and how it shows that it is unlikely that this class of concepts is polynomially learnable from random examples alone in the PAC framework of Valiant. On the other hand, we show that for any fixed bound on the number of objects per scene, this class is polynomially learnable if, in addition to providing random examples, we allow the learning algorithm to make subset queries. In establishing this result, we calculate the capacity of the hypothesis space of conjunctive concepts in a structural domain and use a general theorem of Vapnik and Chervonenkis. This latter result can also be used to estimate a sample size sufficient for heuristic learning techniques that do not use queries.

[1]  Claude Sammut,et al.  LEARNING CONCEPTS BY ASKING QUESTIONS , 1998 .

[2]  David Haussler,et al.  Learnability and the Vapnik-Chervonenkis dimension , 1989, JACM.

[3]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[4]  Wray L. Buntine,et al.  Induction of Horn Clauses: Methods and the Plausible Generalization Algorithm , 1987, Int. J. Man Mach. Stud..

[5]  Alan Bundy,et al.  An Analytical Comparison of Some Rule-Learning Programs , 1985, Artif. Intell..

[6]  David Haussler,et al.  ɛ-nets and simplex range queries , 1987, Discret. Comput. Geom..

[7]  Tom M. Mitchell,et al.  Explanation-Based Generalization: A Unifying View , 1986, Machine Learning.

[8]  Joan Feigenbaum,et al.  Factorization in Experiment Generation , 1986, AAAI.

[9]  Robert E. Stepp.  Machine Learning from Structured Objects , 1987 .

[10]  Barr and Feigenbaum Edward A. Avron.  The Handbook of Artificial Intelligence , 1981 .

[11]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[12]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[13]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[14]  S. Vera,et al.  Induction of Concepts in the Predicate Calculus , 1975, IJCAI.

[15]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[16]  Thomas G. Dietterich,et al.  A Comparative Review of Selected Methods for Learning from Examples , 1983 .

[17]  D. Haussler.  Learning Conjunctive Concepts in Structural Domains , 1989 .

[18]  David Haussler.  Learning Conjunctive Concepts in Structural Domains , 1989, Mach. Learn..

[19]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[20]  HausslerDavid,et al.  A general lower bound on the number of examples needed for learning , 1989 .

[21]  Tom M. Mitchell,et al.  Learning improved integrity constraints and schemas from exceptions in databases and knowledge bases , 1986 .

[22]  David Haussler,et al.  Epsilon-nets and simplex range queries , 1986, SCG '86.

[23]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[24]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[25]  Paul E. Utgoff,et al.  Shift of bias for inductive concept learning , 1984 .

[26]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[27]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[28]  Tom M. Mitchell,et al.  The Need for Biases in Learning Generalizations , 2007 .

[29]  Dana Angluin,et al.  Queries and concept learning , 1988, Machine Learning.

[30]  B. Natarajan.  On learning sets and functions , 2004, Machine Learning.

[31]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[32]  Frederick Hayes-Roth,et al.  An interference matching technique for inducing abstractions , 1978, CACM.

[33]  M. Kearns,et al.  Recent Results on Boolean Concept Learning , 1987 .

[34]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[35]  Judea Pearl,et al.  ON THE CONNECTION BETWEEN THE COMPLEXITY AND CREDIBILITY OF INFERRED MODELS , 1978 .

[36]  John Gill,et al.  Computational Complexity of Probabilistic Turing Machines , 1977, SIAM J. Comput..

[37]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[38]  Ryszard S. Michalski,et al.  A Theory and Methodology of Inductive Learning , 1983, Artificial Intelligence.

[39]  Nils J. Nilsson,et al.  Artificial Intelligence , 1974, IFIP Congress.

[40]  Tom M. Mitchell,et al.  Learning Improved Integrity Constraints and Schemes From Exceptions in Data and Knowledge Bases , 1985, On Knowledge Base Management Systems.

[41]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.