Brain-Like Emergent Spatial Processing

This is a theoretical, modeling, and algorithmic paper about the spatial aspect of brain-like information processing, modeled by the developmental network (DN) model. The new brain architecture allows the external environment (including teachers) to interact with the sensory ends and the motor ends of the skull-closed brain through development. It does not allow the human programmer to hand-pick extra-body concepts or to handcraft the concept boundaries inside the brain . Mathematically, the brain spatial processing performs real-time mapping from to , through network updates, where the contents of all emerge from experience. Using its limited resource, the brain does increasingly better through experience. A new principle is that the effector ends serve as hubs for concept learning and abstraction. The effector ends serve also as input and the sensory ends serve also as output. As DN embodiments, the Where-What Networks (WWNs) present three major function novelties-new concept abstraction, concept as emergent goals, and goal-directed perception. The WWN series appears to be the first general purpose emergent systems for detecting and recognizing multiple objects in complex backgrounds. Among others, the most significant new mechanism is general-purpose top-down attention.

[1]  E. Callaway.  Local circuits in primary visual cortex of the macaque monkey. , 1998, Annual review of neuroscience.

[2]  Zhuowen Tu,et al.  Image Parsing: Unifying Segmentation, Detection, and Recognition , 2005, International Journal of Computer Vision.

[3]  Leslie G. Ungerleider,et al.  Object vision and spatial vision: two cortical pathways , 1983, Trends in Neurosciences.

[4]  S. Carey.  Précis of The Origin of Concepts , 2011, Behavioral and Brain Sciences.

[5]  W. K. Purves.  Life: The Science of Biology , 1985 .

[6]  D. V. van Essen,et al.  A neurobiological model of visual attention and invariant pattern recognition based on dynamic routing of information , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[7]  Juyang Weng,et al.  Where-What Network 5: Dealing with scales for objects in complex backgrounds , 2011, The 2011 International Joint Conference on Neural Networks.

[8]  John K. Tsotsos,et al.  Modeling Visual Attention via Selective Tuning , 1995, Artif. Intell..

[9]  E. Callaway,et al.  Contributions of individual layer 6 pyramidal neurons to local circuitry in macaque primary visual cortex , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[10]  Richard B. Ivry,et al.  Timing in Eyeblink Classical Conditioning and Timed-Interval Tapping , 1999 .

[11]  Juyang Weng,et al.  Developmental Stereo: Emergence of Disparity Preference in Models of the Visual Cortex , 2009, IEEE Transactions on Autonomous Mental Development.

[12]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[13]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[14]  T. Shultz,et al.  Modeling developmental cognitive neuroscience , 2006, Trends in Cognitive Sciences.

[15]  M. Domjan.  The principles of learning and behavior, 4th ed. , 1998 .

[16]  Pietro Perona,et al.  One-shot learning of object categories , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  John R. Anderson,et al.  Rules of the Mind , 1993 .

[18]  Pieter R. Roelfsema,et al.  Attention-Gated Reinforcement Learning of Internal Representations for Classification , 2005, Neural Computation.

[19]  Marvin Minsky,et al.  Logical Versus Analogical or Symbolic Versus Connectionist or Neat Versus Scruffy , 1991, AI Mag..

[20]  S. Grossberg,et al.  Contrast-sensitive perceptual grouping and object-based attention in the laminar circuits of primary visual cortex , 2000, Vision Research.

[21]  G. F. Cooper,et al.  Development of the Brain depends on the Visual Environment , 1970, Nature.

[22]  T. Sejnowski,et al.  Irresistible environment meets immovable neurons , 1997, Behavioral and Brain Sciences.

[23]  C. Koch,et al.  Explicit Encoding of Multimodal Percepts by Single Neurons in the Human Brain , 2009, Current Biology.

[24]  S. Grossberg,et al.  View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based attentional shrouds , 2009, Cognitive Psychology.

[25]  Gary Cantor,et al.  On the Relationship Between Implicit and Explicit Modes in the Learning of a Complex Rule Structure , 1980 .

[26]  M. Sur,et al.  Patterning and Plasticity of the Cerebral Cortex , 2005, Science.

[27]  J. Weng,et al.  Optimal In-Place Self-Organization for Cortical Development : Limited Cells , Sparse Coding and Cortical Topography , 2006 .

[28]  Rama Chellappa,et al.  PADS: A Probabilistic Activity Detection Framework for Video Data , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Juyang Weng,et al.  Top–Down Connections in Self-Organizing Hebbian Networks: Topographic Class Grouping , 2010, IEEE Transactions on Autonomous Mental Development.

[30]  Juyang Weng,et al.  Symbolic Models and Emergent Models: A Review , 2012, IEEE Transactions on Autonomous Mental Development.

[31]  Terrence J. Sejnowski,et al.  What Are the Projective Fields of Cortical Neurons , 2006 .

[32]  Risto Miikkulainen,et al.  Self-organization of hierarchical visual maps with feedback connections , 2006, Neurocomputing.

[33]  Juyang Weng,et al.  Where What Network 3 : Developmental Top-Down Attention with Multiple Meaningful Foregrounds , 2010 .

[34]  M. Sur,et al.  Rewiring cortex: the role of patterned activity in development and plasticity of neocortical circuits. , 1999, Journal of neurobiology.

[35]  Juyang Weng,et al.  Where-what network 1: “Where” and “what” assist each other through top-down connections , 2008, 2008 7th IEEE International Conference on Development and Learning.

[36]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[37]  Paul J. Werbos,et al.  The Roots of Backpropagation: From Ordered Derivatives to Neural Networks and Political Forecasting , 1994 .

[38]  G. Deco,et al.  A hierarchical neural system with attentional top–down enhancement of the spatial resolution for object recognition , 2000, Vision Research.

[39]  W. Lippe,et al.  Rhythmic spontaneous activity in the developing avian auditory system , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[40]  Juyang Weng,et al.  Dually Optimal Neuronal Layers: Lobe Component Analysis , 2009, IEEE Transactions on Autonomous Mental Development.

[41]  D C Van Essen,et al.  Shifter circuits: a computational strategy for dynamic aspects of visual processing. , 1987, Proceedings of the National Academy of Sciences of the United States of America.

[42]  John H. Flavell,et al.  Cognitive development: Past, present, and future. , 1992 .

[43]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[44]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[45]  G. Bi,et al.  Synaptic modification by correlated activity: Hebb's postulate revisited. , 2001, Annual review of neuroscience.

[46]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[47]  Feng-Hsiung Hsu,et al.  IBM's Deep Blue Chess grandmaster chips , 1999, IEEE Micro.

[48]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[49]  Juyang Weng,et al.  Motor initiated expectation through top-down connections as abstract context in a physical world , 2008, 2008 7th IEEE International Conference on Development and Learning.

[50]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[51]  G. Edelman,et al.  Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. , 1998, Cerebral cortex.

[52]  R. Shepard,et al.  Mental Rotation of Three-Dimensional Objects , 1971, Science.

[53]  Justin L. Vincent,et al.  Spontaneous neuronal activity distinguishes human dorsal and ventral attention systems. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[54]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[55]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[56]  E. Knudsen.  Fundamental components of attention. , 2007, Annual review of neuroscience.

[57]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[58]  Anil K. Jain,et al.  Vehicle Segmentation and Classification Using Deformable Templates , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[59]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[60]  Fei-Fei Li,et al.  Modeling mutual context of object and human pose in human-object interaction activities , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[61]  Juyang Weng,et al.  Complex text processing by the temporal context machines , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[62]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[63]  Juyang Weng,et al.  2008 Special issue , 2008 .

[64]  Rajesh P. N. Rao,et al.  An Active Vision Architecture Based on Iconic Representations , 1995, Artif. Intell..

[65]  Narendra Ahuja,et al.  Cresceptron: a self-organizing neural network which grows adaptively , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[66]  M. Domjan.  The principles of learning and behavior , 1982 .

[67]  Juyang Weng,et al.  Task Muddiness, Intelligence Metrics, and the Necessity of Autonomous Mental Development , 2009, Minds and Machines.

[68]  F. Werblin,et al.  Requirement for Cholinergic Synaptic Transmission in the Propagation of Spontaneous Retinal Waves , 1996, Science.

[69]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[70]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[71]  Juyang Weng,et al.  Three theorems: Brain-like networks logically reason and optimally generalize , 2011, The 2011 International Joint Conference on Neural Networks.

[72]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[73]  Risto Miikkulainen,et al.  Computational Maps in the Visual Cortex , 2005 .

[74]  Juyang Weng,et al.  WWN-text: Cortex-like language acquisition with “what” and “where” , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[75]  Narendra Ahuja,et al.  Learning Recognition and Segmentation Using the Cresceptron , 1997, International Journal of Computer Vision.

[76]  J. Piaget.  The construction of reality in the child , 1954 .

[77]  U. Lindenberger,et al.  Cognitive Development , 2014, Front. Young Minds..

[78]  M. Tarr,et al.  Visual Object Recognition , 1996, ISTCS.

[79]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[80]  Michael Cole,et al.  THE DEVELOPMENT OF CHILDREN. , 1925, Science.

[81]  Michael J. O'Donovan.  The origin of spontaneous activity in developing networks of the vertebrate nervous system , 1999, Current Opinion in Neurobiology.

[82]  T. Poggio,et al.  Neural mechanisms of object recognition , 2002, Current Opinion in Neurobiology.

[83]  Juyang Weng,et al.  Laterally connected lobe component analysis: Precision and topography , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[84]  D. Mumford,et al.  Neural activity in early visual cortex reflects behavioral experience and higher-order perceptual saliency , 2002, Nature Neuroscience.

[85]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[86]  J. Elman,et al.  Rethinking Innateness: A Connectionist Perspective on Development , 1996 .

[87]  Juyang Weng,et al.  Optimal In-Place Learning and the Lobe Component Analysis , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[88]  Y. Dan,et al.  Spike timing-dependent plasticity: from synapse to perception. , 2006, Physiological reviews.

[89]  D. Fitzpatrick.  Seeing beyond the receptive field in primary visual cortex , 2000, Current Opinion in Neurobiology.

[90]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[91]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[92]  Juyang Weng,et al.  A 5-chunk developmental brain-mind network model for multiple events in complex backgrounds , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[93]  Juyang Weng,et al.  From neural networks to the brain: autonomous mental development , 2006, IEEE Computational Intelligence Magazine.

[94]  William P Hetrick,et al.  Comparison of auditory and visual conditioning stimuli in delay eyeblink conditioning in healthy young adults , 2009, Learning & behavior.

[95]  Juyang Weng,et al.  Synapse maintenance in the Where-What Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[96]  Alessandra Angelucci,et al.  Induction of visual orientation modules in auditory cortex , 2000, Nature.

[97]  Juyang Weng,et al.  WWN-2: A biologically inspired neural network for concurrent visual attention and recognition , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[98]  Xi Zhang,et al.  Accounting for a variety of reasoning data within a cognitive architecture , 2006, J. Exp. Theor. Artif. Intell..

[99]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[100]  L. Parsons.  Imagined spatial transformations of one's hands and feet , 1987, Cognitive Psychology.

[101]  E. Rolls,et al.  A Neurodynamical cortical model of visual attention and invariant object recognition , 2004, Vision Research.

[102]  S. Shi,et al.  Specific synapses develop preferentially among sister excitatory neurons in the neocortex , 2009, Nature.

[103]  R. Sun,et al.  The interaction of the explicit and the implicit in skill learning: a dual-process approach. , 2005, Psychological review.

[104]  D. Ansari.  Effects of development and enculturation on number representation in the brain , 2008, Nature Reviews Neuroscience.

[105]  Juyang Weng,et al.  Visual Learning with Navigation as an Example , 2000, IEEE Intell. Syst..

[106]  M. Stryker,et al.  The role of visual experience in the development of columns in cat visual cortex. , 1998, Science.