Learning Conjunctive Concepts in Structural Domains

We study the problem of learning conjunctive concepts from examples on structural domains like the blocks world. nus class of concepts is formally defined and it is shown that even for samples in which each example (positive or negative) is a two-object scene it is NP-complete to determine if there is any concept in this class that is consistent with the sample. We demonstrate how this result affects the feasibility of Mitchell's version space approach and how it shows that it is unlikely that this class of concepts is polynomiaIIy learnable from random examples in the sense of Valiant. On the other hand, we show that this class is polynomially learnable if we allow a larger hypothesis space. This result holds for any fixed number of objects per scene, but the algorithm is not practical unless the number of objects per scene is very small. We also show that heuristic methods for learning from larger scenes are likely to give an accurate hypothesis if they produce a simple hypothesis consistent with a large enough random sample.

[1]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[2]  David Haussler,et al.  What Size Net Gives Valid Generalization? , 1989, Neural Computation.

[3]  Tom M. Mitchell,et al.  Learning improved integrity constraints and schemas from exceptions in databases and knowledge bases , 1986 .

[4]  S. Vere.  Induction of concepts in the predicate calculus , 1975, IJCAI 1975.

[5]  Vladimir Vapnik,et al.  Chervonenkis: On the uniform convergence of relative frequencies of events to their probabilities , 1971 .

[6]  Paul E. Utgoff,et al.  Shift of bias for inductive concept learning , 1984 .

[7]  Tom M. Mitchell,et al.  Generalization as Search , 2002 .

[8]  Patrick Henry Winston,et al.  Learning structural descriptions from examples , 1970 .

[9]  David Haussler.  Quantifying the Inductive Bias in Concept Learning (Extended Abstract) , 1986, AAAI.

[10]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[11]  Leslie G. Valiant,et al.  Computational limitations on learning from examples , 1988, JACM.

[12]  Alan Bundy,et al.  An Analytical Comparison of Some Rule-Learning Programs , 1985, Artif. Intell..

[13]  Wray L. Buntine,et al.  Induction of Horn Clauses: Methods and the Plausible Generalization Algorithm , 1987, Int. J. Man Mach. Stud..

[14]  Thomas G. Dietterich,et al.  A Comparative Review of Selected Methods for Learning from Examples , 1983 .

[15]  Ryszard S. Michalski,et al.  Pattern Recognition as Rule-Guided Inductive Inference , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[17]  M MitchellTom,et al.  Explanation-Based Generalization , 1986 .

[18]  David Haussler,et al.  Classifying learnable geometric concepts with the Vapnik-Chervonenkis dimension , 1986, STOC '86.

[19]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[20]  Tom M. Mitchell,et al.  Learning Improved Integrity Constraints and Schemes From Exceptions in Data and Knowledge Bases , 1985, On Knowledge Base Management Systems.

[21]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[22]  David Haussler,et al.  Equivalence of models for polynomial learnability , 1988, COLT '88.

[23]  David Haussler.  Bias, Version Spaces and Valiant's Learning Framework , 1987 .

[24]  Leslie G. Valiant,et al.  Learning Disjunction of Conjunctions , 1985, IJCAI.

[25]  D. Angluin.  Queries and Concept Learning , 1988 .

[26]  David Haussler,et al.  ɛ-nets and simplex range queries , 1987, Discret. Comput. Geom..

[27]  Joan Feigenbaum,et al.  Factorization in Experiment Generation , 1986, AAAI.

[28]  Judea Pearl,et al.  ON THE CONNECTION BETWEEN THE COMPLEXITY AND CREDIBILITY OF INFERRED MODELS , 1978 .

[29]  John Gill,et al.  Computational Complexity of Probabilistic Turing Machines , 1977, SIAM J. Comput..

[30]  Robert E. Stepp.  Machine Learning from Structured Objects , 1987 .

[31]  Frederick Hayes-Roth,et al.  An interference matching technique for inducing abstractions , 1978, CACM.

[32]  M. Kearns,et al.  Recent Results on Boolean Concept Learning , 1987 .