A Model for Auto-Programming for General Purposes

The Universal Turing Machine (TM) is a model for VonNeumann computers --- general-purpose computers. A human brain can inside-skull-automatically learn a universal TM so that he acts as a general-purpose computer and writes a computer program for any practical purposes. It is unknown whether a machine can accomplish the same. This theoretical work shows how the Developmental Network (DN) can accomplish this. Unlike a traditional TM, the TM learned by DN is a super TM --- Grounded, Emergent, Natural, Incremental, Skulled, Attentive, Motivated, and Abstractive (GENISAMA). A DN is free of any central controller (e.g., Master Map, convolution, or error back-propagation). Its learning from a teacher TM is one transition observation at a time, immediate, and error-free until all its neurons have been initialized by early observed teacher transitions. From that point on, the DN is no longer error-free but is always optimal at every time instance in the sense of maximal likelihood, conditioned on its limited computational resources and the learning experience. This letter also extends the Church-Turing thesis to automatic programming for general purposes and sketchily proved it.

[1]  Marvin Minsky,et al.  Logical Versus Analogical or Symbolic Versus Connectionist or Neat Versus Scruffy , 1991, AI Mag..

[2]  J. C. Martin,et al.  Introduction to Languages and the Theory of Computation" 3rd Ed , 1991 .

[3]  Bong-Kiun Kaang,et al.  Multiple repressive mechanisms in the hippocampus during memory formation , 2015, Science.

[4]  Juyang Weng,et al.  Dually Optimal Neuronal Layers: Lobe Component Analysis , 2009, IEEE Transactions on Autonomous Mental Development.

[5]  M. Raijmakers.  Rethinking innateness: A connectionist perspective on development. , 1997 .

[6]  Kunihiko Fukushima,et al.  Cognitron: A self-organizing multilayered neural network , 1975, Biological Cybernetics.

[7]  D C Van Essen,et al.  Shifter circuits: a computational strategy for dynamic aspects of visual processing. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Anne Treisman,et al.  Features and objects in visual processing , 1986 .

[9]  G. F. Cooper,et al.  Development of the Brain depends on the Visual Environment , 1970, Nature.

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  F. Werblin,et al.  Requirement for Cholinergic Synaptic Transmission in the Propagation of Spontaneous Retinal Waves , 1996, Science.

[12]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[13]  T. Sejnowski,et al.  Irresistible environment meets immovable neurons , 1997, Behavioral and Brain Sciences.

[14]  Leila Reddy,et al.  Top–down biases win against focal attention in the fusiform face area , 2007, NeuroImage.

[15]  D. Hubel,et al.  Comparison of the effects of unilateral and bilateral eye closure on cortical unit responses in kittens. , 1965, Journal of neurophysiology.

[16]  Trevor Bekolay,et al.  A Large-Scale Model of the Functioning Brain , 2012, Science.

[17]  M. Sur,et al.  Visual behaviour mediated by retinal projections directed to the auditory pathway , 2000, Nature.

[18]  Giovanni Soda,et al.  Unified Integration of Explicit Knowledge and Learning by Example in Recurrent Networks , 1995, IEEE Trans. Knowl. Data Eng..

[19]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[20]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[21]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[22]  Juyang Weng,et al.  A bridge-islands model for brains: Developing numeric circuits for logic and motivation , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).

[23]  Juyang Weng,et al.  Cross-domain and within-domain synaptic maintenance for autonomous development of visual areas , 2015, 2015 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[24]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[25]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[26]  M. Moser,et al.  A prefrontal–thalamo–hippocampal circuit for goal-directed spatial navigation , 2015, Nature.

[27]  Varol Akman,et al.  Turing Test: 50 Years Later , 2000, Minds and Machines.

[28]  Frank K. Soong,et al.  High performance connected digit recognition, using hidden Markov models , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[29]  E. Miller,et al.  Response to Comment on "Top-Down Versus Bottom-Up Control of Attention in the Prefrontal and Posterior Parietal Cortices" , 2007, Science.

[30]  Juyang Weng,et al.  Synapse maintenance in the Where-What Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[31]  Juyang Weng,et al.  WWN-2: A biologically inspired neural network for concurrent visual attention and recognition , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[32]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[33]  Ron Sun,et al.  The importance of cognitive architectures: an analysis based on CLARION , 2007, J. Exp. Theor. Artif. Intell..

[34]  James S. Albus,et al.  A model of computation and representation in the brain , 2010, Inf. Sci..

[35]  Takayuki Ito,et al.  Neocognitron: A neural network model for a mechanism of visual pattern recognition , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[36]  M. Posner,et al.  Attention and the detection of signals. , 1980, Journal of experimental psychology.

[37]  Juyang Weng,et al.  Where What Network 3 : Developmental Top-Down Attention with Multiple Meaningful Foregrounds , 2010 .

[38]  P. Voss.  Sensitive and critical periods in visual sensory deprivation , 2013, Front. Psychol..

[39]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[40]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[41]  Itamar Arel,et al.  Beyond the Turing Test , 2009, Computer.

[42]  Leslie G. Valiant,et al.  A neuroidal architecture for cognitive computation , 1998, ICALP.

[43]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[44]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[45]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[46]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[47]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[48]  C. Gilbert,et al.  Perceptual learning and top-down influences in primary visual cortex , 2004, Nature Neuroscience.

[49]  R. Sun,et al.  The interaction of the explicit and the implicit in skill learning: a dual-process approach. , 2005, Psychological review.

[50]  Noam Chomsky,et al.  Rules and representations , 1980, Behavioral and Brain Sciences.

[51]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[52]  Juyang Weng,et al.  Why Have We Passed “ Neural Networks Do Not Abstract Well ” ? , 2011 .

[53]  J. Haldane.  The interaction of nature and nurture. , 1946, Annals of eugenics.

[54]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[55]  Jeffrey D. Ullman,et al.  Introduction to Automata Theory, Languages and Computation , 1979 .

[56]  M. Gluck,et al.  Learning and Memory: From Brain to Behavior , 2007 .

[57]  Sanjeev Arora,et al.  Polynomial time approximation schemes for Euclidean traveling salesman and other geometric problems , 1998, JACM.

[58]  L. Vygotsky,et al.  Thought and Language , 1963 .

[59]  Jonathan Krause,et al.  A Hierarchical Approach for Generating Descriptive Image Paragraphs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[61]  C. Lee Giles,et al.  Constructing deterministic finite-state automata in recurrent neural networks , 1996, JACM.

[62]  J. Bruner,et al.  The role of tutoring in problem solving. , 1976, Journal of child psychology and psychiatry, and allied disciplines.

[63]  Juyang Weng,et al.  Brain as an Emergent Finite Automaton: A Theory and Three Theorems , 2015 .

[64]  Juyang Weng,et al.  A computational developmental model for specificity and transfer in perceptual learning. , 2013, Journal of vision.

[65]  Juyang Weng,et al.  Brain-Inspired Concept Networks: Learning Concepts from Cluttered Scenes , 2014, IEEE Intelligent Systems.

[66]  Mark H. Johnson,et al.  Innateness and Emergentism , 2008 .

[67]  P. Frasconi,et al.  Representation of Finite State Automata in Recurrent Radial Basis Function Networks , 1996, Machine Learning.

[68]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[69]  Juyang Weng,et al.  Symbolic Models and Emergent Models: A Review , 2012, IEEE Transactions on Autonomous Mental Development.

[70]  Narendra Ahuja,et al.  Cresceptron: a self-organizing neural network which grows adaptively , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[71]  A. Turing.  On Computable Numbers, with an Application to the Entscheidungsproblem. , 1937 .

[72]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[73]  H T Siegelmann,et al.  Dating and Context of Three Middle Stone Age Sites with Bone Points in the Upper Semliki Valley, Zaire , 2007 .

[74]  Vincent C. Müller.  The hard and easy grounding problems (Comment on A. Cangelosi) , 2011 .

[75]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[76]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[77]  C. Shatz,et al.  Synaptic Activity and the Construction of Cortical Circuits , 1996, Science.

[78]  M. Sur,et al.  Patterning and Plasticity of the Cerebral Cortex , 2005, Science.

[79]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[80]  Leslie G. Valiant,et al.  A theory of the learnable , 1984, CACM.

[81]  Narendra Ahuja,et al.  Learning Recognition and Segmentation Using the Cresceptron , 1997, International Journal of Computer Vision.

[82]  J. Piaget.  The construction of reality in the child , 1954 .

[83]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[84]  Ivan N Pigarev,et al.  Neural Mechanisms of Visual Attention: How Top-Down Feedback Highlights Relevant Locations , 2007, Science.

[85]  S. Pinker.  How the Mind Works , 1999, Philosophy after Darwin.

[86]  Y. Dan,et al.  Spike timing-dependent plasticity: from synapse to perception. , 2006, Physiological reviews.

[87]  Peter Dayan,et al.  Bee foraging in uncertain environments using predictive hebbian learning , 1995, Nature.

[88]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[89]  D. Fitzpatrick,et al.  The development of direction selectivity in ferret visual cortex requires early visual experience , 2006, Nature Neuroscience.

[90]  Douglas B. Lenat,et al.  CYC: a large-scale investment in knowledge infrastructure , 1995, CACM.

[91]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[92]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[93]  Stevan Harnad.  The Symbol Grounding Problem , 1999, ArXiv.

[94]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[95]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.