Induction Over the Unexplained: Using Overly-General Domain Theories to Aid Concept Learning

This paper describes and evaluates an approach to combining empirical and explanation-based learning called Induction Over the Unexplained (IOU). IOU is intended for learning concepts that can be partially explained by an overly-general domain theory. An eclectic evaluation of the method is presented which includes results from all three major approaches: empirical, theoretical, and psychological. Empirical results show that IOU is effective at refining overly-general domain theories and that it learns more accurate concepts from fewer examples than a purely empirical approach. The application of theoretical results from PAC learnability theory explains why IOU requires fewer examples. IOU is also shown to be able to model psychological data demonstrating the effect of background knowledge on human learning.

[1]  Raymond J. Mooney,et al.  Symbolic and Neural Learning Algorithms: An Experimental Comparison , 1991, Machine Learning.

[2]  Michael R. Lowry,et al.  Learning Physical Descriptions From Functional Definitions, Examples, and Precedents , 1983, AAAI.

[3]  Tom M. Mitchell,et al.  Explanation-Based Generalization: A Unifying View , 1986, Machine Learning.

[4]  Robert Earl Stepp,et al.  Conjunctive Conceptual Clustering: A Methodology and Experimentation , 1987 .

[5]  D. Medin,et al.  The role of theories in conceptual coherence. , 1985, Psychological review.

[6]  Ryszard S. Michalski,et al.  A Theory and Methodology of Inductive Learning , 1983, Artificial Intelligence.

[7]  P. Langley,et al.  Production system models of learning and development , 1987 .

[8]  Brian Falkenhainer,et al.  Analogical Processing: A Simulation and Empirical Corroboration , 1987, AAAI.

[9]  Michael J. Pazzani,et al.  Creating a memory of causal relationships - an integration of empirical and explanation-based learning methods , 1990 .

[10]  Gerald DeJong,et al.  Explanation-Based Learning: An Alternative View , 2005, Machine Learning.

[11]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[12]  Allen Ginsberg,et al.  Theory Reduction, Theory Revision, and Retranslation , 1990, AAAI.

[13]  Steven Minton,et al.  Quantitative Results Concerning the Utility of Explanation-based Learning , 1988, Artif. Intell..

[14]  Jude Shavlik,et al.  Refinement ofApproximate Domain Theories by Knowledge-Based Neural Networks , 1990, AAAI.

[15]  David Haussler,et al.  Quantifying Inductive Bias: AI Learning Algorithms and Valiant's Learning Framework , 1988, Artif. Intell..

[16]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[17]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[18]  Pat Langley,et al.  Machine learning as an experimental science , 2004, Machine Learning.

[19]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[20]  Michael J. Pazzani,et al.  Average Case Analysis of Conjunctive Learning Algorithms , 1990, ML.

[21]  David C. Wilkins,et al.  Knowledge Base Refinement Using Apprenticeship Learning Techniques , 1988, AAAI.

[22]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[23]  Leslie G. Valiant,et al.  A general lower bound on the number of examples needed for learning , 1988, COLT '88.

[24]  Gerald DeJong,et al.  Schema Acquisition from One Example: Psychological Evidence for Explanation-Based Learning. , 1987 .

[25]  Douglas H. Fisher,et al.  Knowledge Acquisition Via Incremental Conceptual Clustering , 1987, Machine Learning.

[26]  Thomas G. Dietterich,et al.  Learning and Inductive Inference , 1982 .

[27]  Andrea Pohoreckyj Danyluk,et al.  Finding New Rules for Incomplete Theories: Explicit Biases for Induction with Contextual Information , 1989, ML.

[28]  William W. Cohen.  Learning from Textbook Knowledge: A Case Study , 1990, AAAI.

[29]  Francesco Bergadano,et al.  A Knowledge Intensive Approach to Concept Induction , 1988, ML Workshop.

[30]  Pat Langley,et al.  Toward a Unified Science of Machine Learning , 1989, Machine Learning.

[31]  Paul E. Utgoff,et al.  Incremental Induction of Decision Trees , 1989, Machine Learning.

[32]  H. Hirsh.  Incremental Version-Space Merging: A General Framework for Concept Learning , 1990 .

[33]  Ray Bareiss,et al.  Concept Learning and Heuristic Classification in WeakTtheory Domains , 1990, Artif. Intell..

[34]  Michael John Pazzani.  Learning causal relationships: an integration of empirical and explanation-based learning methods , 1988 .

[35]  Robert J. Hall,et al.  Learning by Failing to Explain: Using Partial Explanations to Learn in Incomplete or Intractable Domains , 2005, Machine Learning.

[36]  Jean H. Gallier,et al.  Linear-Time Algorithms for Testing the Satisfiability of Propositional Horn Formulae , 1984, J. Log. Program..

[37]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[38]  John Mingers,et al.  An Empirical Comparison of Pruning Methods for Decision Tree Induction , 1989, Machine Learning.

[39]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[40]  Ryszard S. Michalski,et al.  Constraints and Preferences in Inductive Learning: An Experimental Study of Human and Machine Performance , 1987 .

[41]  Raymond J. Mooney,et al.  Changing the Rules: A Comprehensive Approach to Theory Refinement , 1990, AAAI.

[42]  Allen Ginsberg,et al.  Theory Revision via Prior Operationalization , 1988, AAAI.

[43]  M. Kearns,et al.  Recent Results on Boolean Concept Learning , 1987 .

[44]  Barr and Feigenbaum Edward A. Avron,et al.  The Handbook of Artificial Intelligence , 1981 .

[45]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, STOC '84.

[46]  Thomas G. Dietterich,et al.  A Study of Explanation-Based Methods for Inductive Learning , 1989, Machine Learning.

[47]  Allen Newell,et al.  Learning by chunking: a production system model of practice , 1987 .

[48]  Michael Lebowitz,et al.  Integrated Learning: Controlling Explanation , 1986, Cogn. Sci..