Issues of Neurodevelopment in Biological and Artificial Neural Networks

In this paper abstractions of neurodevelopmental mechanisms of biological nervous systems are interpreted as changes in the structure, learning parameters and input data of artificial neural networks. A review of neurodevelopmental mechanisms reveals that structural changes follow a typical timeline. We propose it as part of an algorithmic model of neurodevelopment and early learning. We further discuss the concept of incremental learning in artificial neural networks. We conclude that incremental learning can currently only model a small part of neurodevelopment in biology. Neurodevelopment should be modeled in the context of evolution.

[1]  P. Rakić,et al.  Scheduling of monoaminergic neurotransmitter receptor expression in the primate neocortex during postnatal development. , 1992, Cerebral cortex.

[2]  John C. Platt.  A Resource-Allocating Network for Function Interpolation , 1991, Neural Computation.

[3]  Stephan K. Chalup,et al.  First Order Recurrent Neural Networks Learn To Predict A Mildly Context-Sensitive Language , 2000 .

[4]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[5]  Stephen I. Gallant,et al.  Perceptron-based learning algorithms , 1990, IEEE Trans. Neural Networks.

[6]  P. Rakić.  Limits of neurogenesis in primates. , 1985, Science.

[7]  Lutz Prechelt,et al.  Investigation of the CasCor Family of Learning Algorithms , 1997, Neural Networks.

[8]  T. Martínez,et al.  Competitive Hebbian Learning Rule Forms Perfectly Topology Preserving Maps , 1993 .

[9]  Padhraic Smyth,et al.  Discrete recurrent neural networks for grammatical inference , 1994, IEEE Trans. Neural Networks.

[10]  P. Rakić,et al.  Axon overproduction and elimination in the anterior commissure of the developing rhesus monkey , 1994, The Journal of comparative neurology.

[11]  Scott E. Fahlman,et al.  An empirical study of learning speed in back-propagation networks , 1988 .

[12]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[13]  Anders Krogh,et al.  Introduction to the theory of neural computation , 1994, The advanced book program.

[14]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[15]  Douglas L. T. Rohde,et al.  Language acquisition in the absence of explicit negative evidence: how important is starting small? , 1999, Cognition.

[16]  L. Cosmides.  From : The Cognitive Neurosciences , 1995 .

[17]  P. Rakić.  Corticogenesis in human and nonhuman primates. , 1995 .

[18]  Bernd Fritzke,et al.  Growing cell structures--A self-organizing network for unsupervised and supervised learning , 1994, Neural Networks.

[19]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[20]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[21]  P. Rakić,et al.  Axon overproduction and elimination in the corpus callosum of the developing rhesus monkey , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  S. Griffis.  EDITOR , 1997, Journal of Navigation.

[23]  Omid M. Omidvar.  Progress in neural networks , 1991 .

[24]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[25]  C. Gallistel.  The Replacement of General-Purpose Learning Models with Adaptively Specialized Learning Modules , 2000 .

[26]  Byoung-Tak Zhang,et al.  An incremental learning algorithm that optimizes network size and sample size in one trial , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[27]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[28]  Philip T. Quinlan,et al.  Structural change and development in real and artificial neural networks , 1998, Neural Networks.

[29]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[30]  M. Gazzaniga,et al.  The new cognitive neurosciences , 2000 .

[31]  S. Schultz.  Principles of Neural Science, 4th ed. , 2001 .

[32]  P. Huttenlocher,et al.  The development of synapses in striate cortex of man. , 1987, Human neurobiology.

[33]  T. Sejnowski,et al.  Irresistible environment meets immovable neurons , 1997, Behavioral and Brain Sciences.

[34]  J. A. Campos-Ortega.  Ontogenie des Nervensystems und der Sinnesorgane , 2001 .

[35]  Martin A. Riedmiller,et al.  Advanced supervised learning in multi-layer perceptrons — From backpropagation to adaptive learning algorithms , 1994 .

[36]  Elissa L. Newport,et al.  Maturational Constraints on Language Learning , 1990, Cogn. Sci..

[37]  S. Kirby,et al.  The evolution of incremental learning: language, development and critical periods , 1997 .

[38]  Ross F. Hayward.  Analytic and inductive learning in an efficient connectionist rule-based reasoning system , 2001 .

[39]  Yaser S. Abu-Mostafa,et al.  The Vapnik-Chervonenkis Dimension: Information versus Complexity in Learning , 1989, Neural Computation.

[40]  P. Goldman-Rakic,et al.  Concurrent overproduction of synapses in diverse regions of the primate cerebral cortex. , 1986, Science.

[41]  Wirt Atmar,et al.  Notes on the simulation of evolution , 1994, IEEE Trans. Neural Networks.