Adaptive Resonance Theory

Computational models based on cognitive and neural systems are now deeply embedded in the standard repertoire of machine learning and data mining methods, with intelligent learning systems enhancing performance in nearly every existing application area. Beyond data mining, this article shows how models based on adaptive resonance theory (ART) may provide entirely new questions and practical solutions for technological applications. ART models carry out hypothesis testing, search, and incremental fast or slow, self-stabilizing learning, recognition, and prediction in response to large nonstationary databases (big data). Three computational examples, each based on the distributed ART neural network, frame questions and illustrate how a learning system (each with no free parameters) may enhance the analysis of large-scale data. Performance of each task is simulated on a common mapping platform, a remote sensing dataset called the Boston Testbed, available online along with open-source system code. Key design elements of ART models and links to software for each system are included. The article further points to future applications for integrative ART-based systems that have already been computationally specified and simulated. New application directions include autonomous robotics, general-purpose machine vision, audition, speech recognition, language acquisition, eye movement control, visual search, figure-ground separation, invariant object recognition, social cognition, object and spatial attention, scene understanding, spacetime integration, episodic memory, navigation, object tracking, system-level analysis of mental disorders, and machine consciousness. Adaptive Resonance Theory Adaptive resonance theory (ART) neural networks model real-time hypothesis testing, search, learning, recognition, and prediction. Since the 1980s, these models of human cognitive information processing have served as computational engines for a variety of neuromorphic technologies (http://techlab.bu.edu/resources/articles/C5). This article points to a broader range of technology transfers that bring new methods to new problem domains. It describes applications of three specific systems, ART knowledge discov© Springer Science+Business Media New York 2016 C. Sammut, G.I. Webb (eds.), Encyclopedia of Machine Learning and Data Mining, DOI 10.1007/978-1-4899-7502-7 6-1 2 Adaptive Resonance Theory ery, self-supervised ART, and biased ART, and summarizes future application areas for largescale, brain-based model systems.

[1]  D. Jameson,et al.  An opponent-process theory of color vision. , 1957, Psychological review.

[2]  M. Posner,et al.  Retention of Abstract Ideas. , 1970 .

[3]  Eve V. Clark,et al.  WHAT'S IN A WORD? ON THE CHILD'S ACQUISITION OF SEMANTICS IN HIS FIRST LANGUAGE , 1973 .

[4]  O. Vinogradova.  Functional Organization of the Limbic System in the Process of Registration of Information: Facts and Hypotheses , 1975 .

[5]  P. H. Schiller.  Central connections of the retinal ON and OFF pathways , 1982, Nature.

[6]  Leslie G. Ungerleider,et al.  Object vision and spatial vision: two cortical pathways , 1983, Trends in Neurosciences.

[7]  S. Grossberg.  Some Psychophysiological and Pharmacological Correlates of a Developmental, Cognitive and Motivational Theory a , 1984, Annals of the New York Academy of Sciences.

[8]  T Hamanaka,et al.  [Neuropsychology of memory]. , 1985, No to shinkei = Brain and nerve.

[9]  S. Grossberg.  Some Psychophysiological and Pharmacological Correlates of A Developmental, Cognitive, and Motivational Theory , 1987 .

[10]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[11]  Stephen Grossberg,et al.  The ART of adaptive pattern recognition by a self-organizing neural network , 1988, Computer.

[12]  S. Grossberg,et al.  Neural dynamics of planned arm movements: emergent invariants and speed-accuracy properties during trajectory formation. , 1988, Psychological review.

[13]  Stephen Grossberg,et al.  Neural dynamics of adaptive timing and temporal discrimination during associative learning , 1989, Neural Networks.

[14]  G. Lynch,et al.  The neurobiology of learning and memory , 1989, Cognition.

[15]  Stephen Grossberg,et al.  ART 3: Hierarchical search using chemical transmitters in self-organizing pattern recognition architectures , 1990, Neural Networks.

[16]  D. Levine.  Introduction to Neural and Cognitive Modeling , 2018 .

[17]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[18]  Stephen Grossberg,et al.  Adaptive neural networks for control of movement trajectories invariant under speed and force rescaling , 1991 .

[19]  R. S. Baxter,et al.  Supervised adaptive resonance networks , 1991, ANNA '91.

[20]  R. Desimone,et al.  A neural mechanism for working and recognition memory in inferior temporal cortex. , 1991, Science.

[21]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[22]  M. Goodale,et al.  Separate visual pathways for perception and action , 1992, Trends in Neurosciences.

[23]  H. Eichenbaum,et al.  Neuronal activity in the hippocampus during delayed non‐match to sample performance in rats: Evidence for hippocampal processing in recognition memory , 1992, Hippocampus.

[24]  S. Grossberg,et al.  A neural network model of adaptively timed reinforcement learning and hippocampal dynamics. , 1992, Brain research. Cognitive brain research.

[25]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[26]  J. Sweatt,et al.  Mechanisms of memory. , 2003, Journal of geriatric psychiatry and neurology.

[27]  S. Grossberg,et al.  Normal and amnesic learning, recognition and memory by a neural model of cortico-hippocampal interactions , 1993, Trends in Neurosciences.

[28]  L. Squire,et al.  The learning of categories: parallel brain systems for item memory and category knowledge. , 1993, Science.

[29]  Michael Anderson,et al.  NIRS: Large scale ART-1 neural architectures for engineering design retrieval , 1994, Neural Networks.

[30]  Gail A. Carpenter,et al.  A distributed outstar network for spatial pattern learning , 1994, Neural Networks.

[31]  Gerald Sommer,et al.  Pattern Recognition by Self-Organizing Neural Networks , 1994 .

[32]  H. Markram,et al.  Redistribution of synaptic efficacy between neocortical pyramidal neurons , 1996, Nature.

[33]  S. Grossberg,et al.  The Hippocampus and Cerebellum in Adaptively Timed Learning, Recognition, and Movement , 1996, Journal of Cognitive Neuroscience.

[34]  R. Desimone,et al.  Neural Mechanisms of Visual Working Memory in Prefrontal Cortex of the Macaque , 1996, The Journal of Neuroscience.

[35]  S. Grossberg,et al.  A neural model of cerebellar learning for arm movement control: cortico-spino-cerebellar dynamics. , 1997, Learning & memory.

[36]  Gail A. Carpenter,et al.  Distributed Learning, Recognition, and Prediction by ART and ARTMAP Neural Networks , 1997, Neural Networks.

[37]  Aijaz A. Baloch,et al.  A neural model of high-level motion processing: Line motion and formotion dynamics , 1997, Vision Research.

[38]  R. Desimone,et al.  Responses of Neurons in Inferior Temporal Cortex during Memory- Guided Visual Search , 1998 .

[39]  S. Grossberg,et al.  Cortical networks for control of voluntary arm movements under variable force conditions. , 1998, Cerebral cortex.

[40]  Gail A. Carpenter,et al.  ARTMAP-IC and medical diagnosis: Instance counting and inconsistent cases , 1998, Neural Networks.

[41]  S. Grossberg.  How does the cerebral cortex work? Learning, attention, and grouping by the laminar circuits of visual cortex. , 1999, Spatial vision.

[42]  S. Grossberg,et al.  A neural model of saccadic eye movement control explains task-specific adaptation , 1999, Vision Research.

[43]  Joshua W. Brown,et al.  How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues , 1999, The Journal of Neuroscience.

[44]  S. Grossberg.  The Link between Brain Learning, Attention, and Consciousness , 1999, Consciousness and Cognition.

[45]  Aijaz A. Baloch,et al.  Neural model of first-order and second-order motion perception and magnocellular dynamics. , 1999, Journal of the Optical Society of America. A, Optics, image science, and vision.

[46]  P. Todd,et al.  Musical networks: Parallel distributed perception and performance , 1999 .

[47]  D A Pollen,et al.  On the neural correlates of visual perception. , 1999, Cerebral cortex.

[48]  S. Grossberg.  The imbalanced brain: from normal behavior to schizophrenia , 2000, Biological Psychiatry.

[49]  M. Page,et al.  Connectionist modelling in psychology: A localist manifesto , 2000, Behavioral and Brain Sciences.

[50]  Elena P. Sapozhnikova,et al.  A modified search procedure for the ART neural networks , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[51]  S. Grossberg,et al.  Neural dynamics of 3-D surface perception: Figure-ground separation and lightness perception , 2000, Perception & psychophysics.

[52]  S. Grossberg.  How hallucinations may arise from brain mechanisms of learning, attention, and volition , 2000, Journal of the International Neuropsychological Society.

[53]  S. Grossberg.  The complementary brain: unifying brain dynamics and modularity , 2000, Trends in Cognitive Sciences.

[54]  Christopher C. Pack,et al.  A Neural Model of Smooth Pursuit Control and Motion Perception by Cortical Area MST , 2001, Journal of Cognitive Neuroscience.

[55]  Stephen Grossberg,et al.  Neural dynamics of motion integration and segmentation within and across apertures , 2001, Vision Research.

[56]  Jacques Verly,et al.  Fusion of multisensor passive and active 3D imagery , 2001, SPIE Defense + Commercial Sensing.

[57]  G. Carpenter.  Neural-network models of learning and memory: leading questions and an emerging framework , 2001, Trends in Cognitive Sciences.

[58]  S. C. Hui,et al.  Neural Networks for Web Content Filtering , 2002, IEEE Intell. Syst..

[59]  Lorenzo Bruzzone,et al.  Image fusion techniques for remote sensing applications , 2002, Inf. Fusion.

[60]  G. Carpenter.  Default ARTMAP , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[61]  S. Sitharama Iyengar,et al.  Adaptive neural network clustering of Web users , 2004, Computer.

[62]  S. Grossberg,et al.  Adaptive pattern classification and universal recoding: I. Parallel development and coding of neural feature detectors , 1976, Biological Cybernetics.

[63]  Stephen Grossberg,et al.  A laminar cortical model for 3D perception of slanted and curved surfaces and of 2D images: development, attention, and bistability , 2004, Vision Research.

[64]  Stephen Grossberg,et al.  How laminar frontal cortex and basal ganglia circuits interact to control planned and reactive saccades , 2004, Neural Networks.

[65]  Stephen Grossberg,et al.  A laminar cortical model of stereopsis and 3D surface perception: closure and da Vinci stereopsis. , 2004, Spatial vision.

[66]  S. Grossberg,et al.  Neural dynamics of autistic behaviors: cognitive, emotional, and timing substrates. , 2006, Psychological review.

[67]  S. Grossberg,et al.  A neural model of 3D shape-from-texture: Multiple-scale filtering, boundary grouping, and surface filling-in , 2007, Vision Research.

[68]  S. Grossberg,et al.  Laminar cortical dynamics of visual form and motion interactions during coherent object motion perception. , 2007, Spatial vision.

[69]  S. Grossberg,et al.  Dopaminergic and non-dopaminergic value systems in conditioning and outcome-specific revaluation , 2008, Brain Research.

[70]  S. Grossberg,et al.  Laminar cortical dynamics of cognitive and motor working memory, sequence learning and performance: toward a unified theory of how the cerebral cortex works. , 2008, Psychological review.

[71]  Dezhao Chen,et al.  An Enhanced ART2 Neural Network for Clustering Analysis , 2008, First International Workshop on Knowledge Discovery and Data Mining (WKDD 2008).

[72]  Stephen Grossberg,et al.  A neural model of visually guided steering, obstacle avoidance, and route selection. , 2009, Journal of experimental psychology. Human perception and performance.

[73]  Stephen Grossberg,et al.  A neural model of how the brain computes heading from optic flow in realistic scenes , 2009, Cognitive Psychology.

[74]  S. Grossberg,et al.  View-invariant object category learning, recognition, and search: How spatial and object attention are coordinated using surface-based attentional shrouds , 2009, Cognitive Psychology.

[75]  Stephen Grossberg,et al.  Cortical dynamics of navigation and steering in natural scenes: Motion-based object segmentation, heading, and obstacle avoidance , 2009, Neural Networks.

[76]  Stephen Grossberg,et al.  ARTSCENE: A neural system for natural scene classification. , 2009, Journal of vision.

[77]  S. Grossberg,et al.  From stereogram to surface: how the brain sees the world in depth. , 2009, Spatial vision.

[78]  Stephen Grossberg,et al.  Target Selection by the Frontal Cortex during Coordinated Saccadic and Smooth Pursuit Eye Movements , 2009, Journal of Cognitive Neuroscience.

[79]  Stephen Grossberg,et al.  How do children learn to follow gaze , share joint attention , imitate their teachers , and use tools during social interactions ? , 2010 .

[80]  Gail A. Carpenter,et al.  Self-supervised ARTMAP , 2010, Neural Networks.

[81]  S. Grossberg,et al.  Cortical dynamics of contextually cued attentive visual learning and search: spatial and object evidence accumulation. , 2010, Psychological review.

[82]  Gail A. Carpenter,et al.  Biased ART: A neural architecture that shifts attention toward previously disregarded features following an incorrect prediction , 2010, Neural Networks.

[83]  S. Grossberg,et al.  How Does the Brain Rapidly Learn and Reorganize View- and Positionally-Invariant Object Representations in Inferior Temporal Cortex? , 2011 .

[84]  Stephen Grossberg,et al.  How Do Object Reference Frames and Motion Vector Decomposition Emerge in Laminar Cortical Circuits? , 2011 .

[85]  Stephen Grossberg,et al.  Neural dynamics of saccadic and smooth pursuit eye movement coordination during visual tracking of unpredictably moving targets , 2012, Neural Networks.

[86]  Stephen Grossberg,et al.  How Do Spatial Learning and Memory Occur in the Brain? Coordinated Learning of Entorhinal Grid Cells and Hippocampal Place Cells , 2012, Journal of Cognitive Neuroscience.

[87]  Stephen Grossberg,et al.  Grid cell hexagonal patterns formed by fast self‐organized learning within entorhinal cortex , 2012, Hippocampus.

[88]  Andreas G. Andreou,et al.  Adaptive Resonance Theory Microchips: Circuit Design Techniques , 2012 .

[89]  Stephen Grossberg,et al.  Joining distributed pattern processing and homeostatic plasticity in recurrent on-center off-surround shunting networks: Noise, saturation, short-term memory, synaptic scaling, and BDNF , 2012, Neural Networks.

[90]  Stephen Grossberg,et al.  Stereopsis and 3D surface perception by spiking neurons in laminar cortical circuits: A method for converting neural rate models into spiking models , 2012, Neural Networks.

[91]  Nicholas C. Foley,et al.  Neural Dynamics of Object-based Multifocal Visual Spatial Attention and Priming: Object Cueing, Useful-field-of-view, and Crowding Cognitive Psychology , 2012 .

[92]  Stephen Grossberg,et al.  How Entorhinal Grid Cells May Learn Multiple Spatial Scales from a Dorsoventral Gradient of Cell Response Rates in a Self-organizing Map , 2012, PLoS Comput. Biol..