Could Knowledge-Based Neural Learning be Useful in Developmental Robotics? The Case of Kbcc

The new field of developmental robotics faces the formidable challenge of implementing effective learning mechanisms in complex, dynamic environments. We make a case that knowledge-based learning algorithms might help to meet this challenge. A constructive neural learning algorithm, knowledge-based cascade-correlation (KBCC), autonomously recruits previously-learned networks in addition to the single hidden units recruited by ordinary cascade-correlation. This enables learning by analogy when adequate prior knowledge is available, learning by induction from examples when there is no relevant prior knowledge, and various combinations of analogy and induction. A review of experiments with KBCC indicates that recruitment of relevant existing knowledge typically speeds learning and sometimes enables learning of otherwise impossible problems. Some additional domains of interest to developmental robotics are identified in which knowledge-based learning seems essential. The characteristics of KBCC in relation to other knowledge-based neural learners and analogical reasoning are summarized as is the neurological basis for learning from knowledge. Current limitations of this approach and directions for future work are discussed.

[1]  J. M. Anglin Vocabulary Development: A Morphological Analysis , 1994 .

[2]  Sebastian Thrun,et al.  Towards programming tools for robots that integrate probabilistic computation and learning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[3]  Rodney A. Brooks,et al.  Elephants don't play chess , 1990, Robotics Auton. Syst..

[4]  R. Sun,et al.  The interaction of the explicit and the implicit in skill learning: a dual-process approach. , 2005, Psychological review.

[5]  J. Tenenbaum,et al.  Special issue on “Probabilistic models of cognition , 2022 .

[6]  K. Holyoak,et al.  Analogical problem solving , 1980, Cognitive Psychology.

[7]  A. Karmiloff-Smith Précis of Beyond modularity: A developmental perspective on cognitive science , 1994, Behavioral and Brain Sciences.

[8]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[9]  Thomas R. Shultz,et al.  Finding relevant knowledge: KBCC applied to DNA splice-junction determination , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[10]  Thomas R. Shultz,et al.  Application of knowledge-based cascade-correlation to vowel recognition , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[11]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[12]  Ellen M. Markman,et al.  Word Learning in Children: An Examination of Fast Mapping. , 1987 .

[13]  Li Fei-Fei Knowledge transfer in learning to recognize visual objects classes , 2006 .

[14]  Karen Wynn,et al.  Addition and subtraction by human infants , 1992, Nature.

[15]  Jude W. Shavlik,et al.  Combining Symbolic and Neural Learning , 1994, Machine Learning.

[16]  Dedre Gentner,et al.  Structure-Mapping: A Theoretical Framework for Analogy , 1983, Cogn. Sci..

[17]  Thomas R. Shultz,et al.  Modeling cognitive development on balance scale phenomena , 1994, Machine Learning.

[18]  R. Poldrack,et al.  Recovering Meaning Left Prefrontal Cortex Guides Controlled Semantic Retrieval , 2001, Neuron.

[19]  Anthony V. Robins,et al.  Transfer in Cognition , 1996, Connect. Sci..

[20]  A. Karmiloff-Smith,et al.  A developmental perspective on cognitive science , 1992 .

[21]  Brenda R. J. Jansen,et al.  Statistical Test of the Rule Assessment Methodology by Latent Class Analysis , 1997 .

[22]  E. Wisniewski,et al.  Prior knowledge and functionally relevant features in concept learning. , 1995, Journal of experimental psychology. Learning, memory, and cognition.

[23]  Jude W. Shavlik,et al.  Extracting refined rules from knowledge-based neural networks , 2004, Machine Learning.

[24]  T. Shultz,et al.  Why let networks grow , 2007 .

[25]  Jonathan Baxter,et al.  Learning internal representations , 1995, COLT '95.

[26]  P. Gordon Level-ordering in lexical development , 1985, Cognition.

[27]  Sandy Lovie How the mind works , 1980, Nature.

[28]  Thomas R. Shultz,et al.  A Compositional Neural-network Solution to Prime-number Testing , 2006 .

[29]  T. Shultz Computational Developmental Psychology , 2003 .

[30]  Gregory Ashby,et al.  A neuropsychological theory of multiple systems in category learning. , 1998, Psychological review.

[31]  Juyang Weng,et al.  Online image classification using IHDR , 2003, International Journal on Document Analysis and Recognition.

[32]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[33]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[34]  Shumeet Baluja,et al.  Reducing Network Depth in the Cascade-Correlation Learning Architecture, , 1994 .

[35]  E. Markman,et al.  Rapid Word Learning in 13- and 18-Month-Olds. , 1994 .

[36]  R. Siegler Three aspects of cognitive development , 1976, Cognitive Psychology.

[37]  Stefano Nolfi,et al.  Adaptation as a more powerful tool than decomposition and integration: experimental evidences from evolutionary robotics , 1998, 1998 IEEE International Conference on Fuzzy Systems Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36228).

[38]  Jordan B. Pollack,et al.  Recursive Distributed Representations , 1990, Artif. Intell..

[39]  Thomas R. Shultz,et al.  Compositionality in a Knowledge-Based Constructive Learner , 2004, AAAI Technical Report.

[40]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1989, IJCAI 1989.

[41]  E. Markman,et al.  Word learning in children: an examination of fast mapping. , 1987, Child development.

[42]  Thomas R. Shultz,et al.  Knowledge-based cascade-correlation: Using knowledge to speed learning , 2001, Connect. Sci..

[43]  Michael Schmitt,et al.  On the Complexity of Computing and Learning with Multiplicative Neural Networks , 2002, Neural Computation.

[44]  E. Heit,et al.  Models of the effects of prior knowledge on category learning. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[45]  Willard Van Orman Quine,et al.  Word and Object , 1960 .

[46]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[47]  Juyang Weng,et al.  Developmental Robotics: Theory and Experiments , 2004, Int. J. Humanoid Robotics.

[48]  S. Pinker The Language Instinct , 1994 .

[49]  E. Spelke Initial knowledge: six suggestions , 1994, Cognition.

[50]  Thomas R. Shultz,et al.  A dual-phase technique for pruning constructive networks , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[51]  G. Murphy A rational theory of concepts , 1993 .

[52]  R. Byrne,et al.  Priming primates: Human and otherwise , 1998, Behavioral and Brain Sciences.

[53]  A. Meyer,et al.  Introduction to Number Theory , 2005 .

[54]  A. Vandierendonck,et al.  A parallel rule activation and rule synthesis model for generalization in category learning , 1995, Psychonomic bulletin & review.

[55]  Tim van Gelder,et al.  Compositionality: A Connectionist Variation on a Classical Theme , 1990, Cogn. Sci..

[56]  A. Clark,et al.  Trading spaces: Computation, representation, and the limits of uninformed learning , 1997, Behavioral and Brain Sciences.

[57]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[58]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[59]  M. Pazzani Influence of prior knowledge on concept acquisition: Experimental and computational results. , 1991 .

[60]  Herbert Hoijtink,et al.  Rules in the balance: Classes, strategies, or rules for the Balance Scale Task? , 2001 .

[61]  Maryellen C. MacDonald,et al.  Language learning and innateness: Some implications of Compounds Research , 2003, Cognitive Psychology.

[62]  Doina Precup,et al.  Combining TD-learning with Cascade-correlation Networks , 2003, ICML.

[63]  I. Biederman,et al.  Dynamic binding in a neural network for shape recognition. , 1992, Psychological review.

[64]  J.-P. Thivierge,et al.  Transferring domain rules in a constructive network: introducing RBCC , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[65]  U. Neisser Concepts and Conceptual Development: Ecological and Intellectual Factors in Categorization , 1989 .

[66]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[67]  Sebastian Thrun,et al.  Integrating Inductive Neural Network Learning and Explanation-Based Learning , 1993, IJCAI.

[68]  Stella Vosniadou,et al.  Simulating Frontotemporal Pathways Involved in Lexical Ambiguity Resolution , 2005 .

[69]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[70]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[71]  Yoshio Takane,et al.  Rule following and rule use in the balance-scale task , 2007, Cognition.

[72]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[73]  J. Kruschke,et al.  Rules and exemplars in category learning. , 1998, Journal of experimental psychology. General.

[74]  I. Biederman Recognition-by-components: a theory of human image understanding. , 1987, Psychological review.

[75]  Brenda R. J. Jansen,et al.  The development of children's rule use on the balance scale task. , 2002, Journal of experimental child psychology.

[76]  Jude Shavlik,et al.  A Framework for Combining Symbolic and Neural Learning , 1992 .

[77]  Thomas R. Shultz,et al.  Using Knowledge to Speed Learning: A Comparison of Knowledge-based Cascade-correlation and Multi-task Learning , 2000, ICML.

[78]  Daniel L. Silver,et al.  The Parallel Transfer of Task Knowledge Using Dynamic Learning Rates Based on a Measure of Relatedness , 1996, Connect. Sci..

[79]  Ellen M. Markman,et al.  Categorization and Naming in Children: Problems of Induction , 1989 .

[80]  Pattie Maes,et al.  Toward the Evolution of Dynamical Neural Networks for Minimally Cognitive Behavior , 1996 .

[81]  K. Nader,et al.  Fear memories require protein synthesis in the amygdala for reconsolidation after retrieval , 2000, Nature.

[82]  S. Quartz Neural networks, nativism, and the plausibility of constructivism , 1993, Cognition.

[83]  Alan Hendrickson,et al.  Hans Jürgen Eysenck (1916–1997) , 1997, Trends in Cognitive Sciences.

[84]  J. Reznick,et al.  Early lexical acquisition: rate, content, and the vocabulary spurt , 1990, Journal of Child Language.

[85]  George A. Miller,et al.  The science of words , 1991 .

[86]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[87]  Robert M. French,et al.  Semi-distributed Representations and Catastrophic Forgetting in Connectionist Networks , 1992 .

[88]  Andreas Wichert,et al.  Pictorial reasoning with cell assemblies , 2001, Connect. Sci..

[89]  K. Holyoak,et al.  Schema induction and analogical transfer , 1983, Cognitive Psychology.

[90]  T. Shultz,et al.  Generative connectionist networks and constructivist cognitive development , 1996 .

[91]  Lorien Y. Pratt,et al.  Discriminability-Based Transfer between Neural Networks , 1992, NIPS.

[92]  G. V. Nakamura Knowledge-based classification of ill-defined categories , 1985, Memory & cognition.

[93]  Karl J. Friston Learning and inference in the brain , 2003, Neural Networks.

[94]  T. Shultz,et al.  Simulating Frontotemporal Pathways Involved in Lexical Ambiguity Resolution , 2005 .

[95]  Rolf Pfeifer,et al.  Understanding intelligence , 2020, Inequality by Design.