Inductive rule learning on the knowledge level

We present an application of the analytical inductive programming system Igor to learning sets of recursive rules from positive experience. We propose that this approach can be used within cognitive architectures to model regularity detection and generalization learning. Induced recursive rule sets represent the knowledge which can produce systematic and productive behavior in complex situations - that is, control knowledge for chaining actions in different, but structural similar situations. We argue, that an analytical approach which is governed by regularity detection in example experience is more plausible than generate-and-test approaches. After introducing analytical inductive programming with Igor we will give a variety of example applications from different problem solving domains. Furthermore, we demonstrate that the same generalization mechanism can be applied to rule acquisition for reasoning and natural language processing.

[1]  S. Ohlsson,et al.  Constraint relaxation and chunk decomposition in insight problem solving , 1999 .

[2]  Michael A. Covington,et al.  Natural Language Processing for Prolog Programmers , 1993 .

[3]  John R. Anderson,et al.  Tower of Hanoi: evidence for the cost of goal retrieval. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[4]  Elliot Soloway,et al.  Studying the Novice Programmer , 1988 .

[5]  Gordon Plotkin,et al.  A Note on Inductive Generalization , 2008 .

[6]  R. Mike Cameron-Jones,et al.  Induction of logic programs: FOIL and related systems , 1995, New Generation Computing.

[7]  Ron Sun,et al.  The importance of cognitive architectures: an analysis based on CLARION , 2007, J. Exp. Theor. Artif. Intell..

[8]  Susumu Katayama Systematic search for lambda expressions , 2005, Trends in Functional Programming.

[9]  Hector Geffner,et al.  Learning Generalized Policies in Planning Using Concept Languages , 2000, KR.

[10]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[11]  Phillip D. Summers,et al.  A Methodology for LISP Program Construction from Examples , 1977, J. ACM.

[12]  H. Simon,et al.  Why are some problems hard? Evidence from Tower of Hanoi , 1985, Cognitive Psychology.

[13]  Allen Newell,et al.  The chunking of goal hierarchies: a generalized model of practice , 1993 .

[14]  Ute Schmid,et al.  Analysis and Evaluation of Inductive Programming Systems in a Higher-Order Framework , 2008, KI.

[15]  Alex S. Taylor,et al.  Machine intelligence , 2009, CHI.

[16]  J. Gregory Trafton,et al.  Memory for goals: an activation-based model , 2002, Cogn. Sci..

[17]  John R. Anderson,et al.  Tracing Problem Solving in Real Time: fMRI Analysis of the Subject-paced Tower of Hanoi , 2005, Journal of Cognitive Neuroscience.

[18]  Alan W. Biermann,et al.  Automatic Program Construction Techniques , 1984 .

[19]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[20]  Thomas G. Dietterich What is machine learning? , 2020, Archives of Disease in Childhood.

[21]  M. Braine,et al.  On learning the grammatical order of words. , 1963, Psychological review.

[22]  Emanuel Kitzelmann,et al.  A Combined Analytical and Search-Based Approach for the Inductive Synthesis of Functional Programs , 2011, KI - Künstliche Intelligenz.

[23]  G. Marcus The Algebraic Mind: Integrating Connectionism and Cognitive Science , 2001 .

[24]  Zohar Manna,et al.  How to clear a block: A theory of plans , 2004, Journal of Automated Reasoning.

[25]  H. Simon,et al.  Models of Bounded Rationality: Empirically Grounded Economic Reason , 1997 .

[26]  Paolo Traverso,et al.  Automated Planning: Theory & Practice , 2004 .

[27]  Noam Chomsky Review of B.F. Skinner, Verbal Behavior , 1959 .

[28]  N. Goodman Fact, Fiction, and Forecast , 1955 .

[29]  Mariëtte Huizinga,et al.  Tower of Hanoi disk-transfer task: Influences of strategy knowledge and learning on performance , 2005 .

[30]  Craig A. Kaplan,et al.  In search of insight , 1990, Cognitive Psychology.

[31]  Allen Newell,et al.  The Knowledge Level , 1989, Artif. Intell..

[32]  Steven Minton,et al.  Selectively Generalizing Plans for Problem-Solving , 1985, IJCAI.

[33]  J. McCarthy Situations, Actions, and Causal Laws , 1963 .

[34]  R. Mayer Thinking, Problem Solving, Cognition , 1983 .

[35]  J. G. Wallace,et al.  Cognitive development: An information-processing view , 1976 .

[36]  S. Phillips,et al.  Processing capacity defined by relational complexity: implications for comparative, developmental, and cognitive psychology. , 1998, The Behavioral and brain sciences.

[37]  José Hernández-Orallo,et al.  Approaches and Applications of Inductive Programming, Third International Workshop, AAIP 2009, Edinburgh, UK, September 4, 2009. Revised Papers , 2010, AAIP.

[38]  C. Cordell Green,et al.  Application of Theorem Proving to Problem Solving , 1969, IJCAI.

[39]  Richard E. Korf,et al.  Macro-Operators: A Weak Method for Learning , 1985, Artif. Intell..

[40]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[41]  Julian M. Pine,et al.  Constructing a Language: A Usage-Based Theory of Language Acquisition. , 2004 .

[42]  Roland Olsson,et al.  Inductive Functional Programming Using Incremental Program Transformation , 1995, Artif. Intell..

[43]  J. Shavlik Acquiring Recursive and Iterative Concepts with Explanation-Based Learning , 1990, Machine Learning.

[44]  Donato Malerba,et al.  Learning Recursive Theories in the Normal ILP Setting , 2003, Fundam. Informaticae.

[45]  B. Karl,et al.  Tatsachen und Probleme zu einer Psychologie der Denkvorgänge, III. , 1908 .

[46]  Martin Hofmann,et al.  I/O guided detection of list catamorphisms: towards problem specific use of program templates in IP , 2010, PEPM '10.

[47]  Fritz Wysotzki,et al.  Applying Inductive Program Synthesis to Macro Learning , 2000, AIPS.

[48]  Jaime G. Carbonell,et al.  Derivational Analogy in PRODIGY: Automating Case Acquisition, Storage, and Utilization , 1993, Machine Learning.

[49]  Ute Schmid,et al.  Inductive Synthesis of Functional Programs: An Explanation Based Generalization Approach , 2006, J. Mach. Learn. Res..

[50]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .

[51]  J. Tenenbaum,et al.  Theory-based Bayesian models of inductive learning and reasoning , 2006, Trends in Cognitive Sciences.

[52]  Willem J. M. Levelt,et al.  What became of LAD , 1975 .

[53]  Ute Schmid,et al.  AAIP 2009 Proceedings of the ACM SIGPLAN Workshop on Approaches and Applications of Inductive Programming , 2009 .

[54]  K. Bühler,et al.  Tatsachen und Probleme zu einer Psychologie der Denkvorgänge: I Über Gedanken , 1907 .

[55]  Stan Szpakowicz,et al.  Review of Natural language processing for prolog programmers by Michael A. Covington. Prentice-Hall 1994. , 1996 .

[56]  Emanuel Kitzelmann,et al.  Analytical Inductive Functional Programming , 2009, LOPSTR.

[57]  J. Gregory Trafton,et al.  Erratum to "Memories for goals: An activation-based model"[Cognitive Science 26 (2002) 39-83] , 2002, Cogn. Sci..

[58]  B. Baars,et al.  How conscious experience and working memory interact , 2003, Trends in Cognitive Sciences.

[59]  Frank Bomarius,et al.  KI 2008: Advances in Artificial Intelligence, 31st Annual German Conference on AI, KI 2008, Kaiserslautern, Germany, September 23-26, 2008. Proceedings , 2008, KI.

[60]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[61]  J. Bach,et al.  Principles of Synthetic Intelligence: Psi: An Architecture of Motivated Cognition , 2009 .

[62]  Yasubumi Sakakibara,et al.  Recent Advances of Grammatical Inference , 1997, Theor. Comput. Sci..

[63]  Pierre Flener,et al.  An introduction to inductive programming , 2008, Artificial Intelligence Review.

[64]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[65]  John R. Anderson,et al.  Tower of Hanoi: evidence for the cost of goal retrieval. , 2001 .

[66]  E. Mark Gold,et al.  Language Identification in the Limit , 1967, Inf. Control..

[67]  Hank Kahney,et al.  What do novice programmers know about recursion , 1983, CHI '83.

[68]  Philip J. Stone,et al.  Experiments in induction , 1966 .

[69]  K. Maccorquodale ON CHOMSKY'S REVIEW OF SKINNER'S VERBAL BEHAVIOR , 1970 .

[70]  Paolo Traverso,et al.  Automated planning - theory and practice , 2004 .

[71]  William M. Smith,et al.  A Study of Thinking , 1956 .

[72]  H A Simon,et al.  The theory of learning by doing. , 1979, Psychological review.

[73]  S. Ohlsson Restructuring revisited: I. Summary and critique of the Gestalt theory of problem solving. , 1984 .

[74]  Pascal Hitzler,et al.  Perspectives of Neural-Symbolic Integration , 2007, Studies in Computational Intelligence.

[75]  John R. Anderson,et al.  MACHINE LEARNING An Artificial Intelligence Approach , 2009 .

[76]  Jaime G. Carbonell,et al.  Towards a General Framework for Composing Disjunctive and Iterative Macro-operators , 1989, IJCAI.