Bring ART into the ACT

ACT is compared with a particular type of connectionist model that cannot handle symbols and use non-biological operations that cannot learn in real time. This focus continues an unfortunate trend of straw man "debates" in cognitive science. Adaptive Resonance Theory, or ART, neural models of cognition can handle both symbols and sub-symbolic representations, and meets the Newell criteria at least as well as these models. COMMENTARY The authors' nomenclature "classical connectionist models" falsely suggests that such models satisfy Newell criteria better than other neural models of cognition. The authors then dichotomize ACT with "classical" connectionism based on its "failure to acknowledge a symbolic level to thought. In contrast, ACT-R includes both symbolic and subsymbolic components" (pp. 1-2). Actually, neural models of cognition such as ART include both types of representation, and clarify how they are learned. Moreover, ART was introduced before the "classical" models (Grossberg, 1976, 1978a, 1980) and naturally satisfies key Newell criteria. In fact, Figures 2 and 3 of ACT are reminiscent of ART circuits (e.g., Carpenter and Grossberg, 1991; Grossberg, 1999b). But ART goes further by proposing how laminar neocmtical circuits integrate bottomup, horizontal, and top-down interactions for intelligent computation (Grossberg, 1999a; Raizada and Grossberg, 2003). Critiques of "classical" connectionist models, below called CM (Carnegie Mellon) connectionism, show that many such models cannot exist in the brain (e.g., Grossberg, 1988; Grossberg et a!., 1997; Grossberg and Merrill, 1996). Below ART is claimed to satisfy many Newell criteria better, with the obvious caveat that no model is yet a complete neural theory of cognition. Flexible behavior: ART models are self-organizing neural production systems capable of fast, stable, real-time learning about arbitrarily large, unexpectedly changing environments (Carpenter and Grossberg, 1991). These properties suit ART for large-scale technological applications, ranging from control of mobile robots, face recognition, remote sensing, medical diagnosis, and electrocardiogram analysis to tool failure monitoring, chemical analysis, circuit design, protein/DNA analysis, musical analysis, and seismic, sonar, and radar recognition, in both software and VLSI microchips (e.g., Carpenter et a!., 1999; Carpenter and Milenova, 2000; Granger eta!., 2001). The criticism ofCM connectionism "that complex, sequentially organized, hierarchical behavior" cannot be modeled also does not apply to ART (e.g., Brad ski et a!., 1994; Cohen and Grossberg, 1978a, 1986; Grossberg and Kuperstein, 1989; Grossberg and Myers, 2000; also see Dynamic Behavior below). Real-time pe1jormance: ART models are manifestly real-time in design, unlike CM connectionist models. Adaptive behavior: ART provides a rigorous solution of the stability-plasticity dilemma, which was my term for catastrophic forgetting before that phrase was coined. "Limitations like shortterm memory" (p. 25) can be derived fi·om the LTM Invariance Principle, which proposes how working memories are designed to enable their stored event sequences to be stably chunked and remembered (Bradski et a!., 1994; Grossberg, 1978a, 1978b ). Vast Knowledge Base: ART can directly access the globally best-matching information in its mem01y no matter how much it has learned. It includes additional criteria of value and temporal relevance through its embedding in START models that include Cognitive-Emotional and Adaptive Timing circuits in addition to cognitive ART circuits (Grossberg and Merrill, 1992, 1996). Dynamic Behavior: "Dealing with dynamic behavior requires a theory of perception and action as well as a the01y of cognition" (p. 6). LAMINART models propose how ART principles are incorporated into perceptual neocortical circuits and how high-level cognitive constraints can modulate lower perceptual representations through top-down matching and attention (Grossberg, 1999a; Raizada and Grossberg, 2003). ART deals with novelty through complementmy interactions between attentional and orienting systems (Grossberg, 1999b, 2000b), the former including cortico-cOJiical and the latter hippocampal circuits. Action circuits also obey laws that are complementmy to those used in perception and cognition (Grossberg, 2000b), notably YAM (Vector Associative Map) laws. V AM-based models have simulated identified brain cells and circuits and the actions that they control (e.g., Brown et a!., 1999; Bullock eta!., 1998; ContrerasVidal et a!., 1997; Fiala et a!., 1996; Gancarz and Grossberg, 1999; Grossberg et a!., 1997), including models of motor skill learning and perf01mance (Bullock, Grossberg, and Guenther, 1993; Bullock, Grossberg, and Mannes, 1993; Grossberg and Paine, 2000). Knowledge Integration: ART reconciles distributed and syn1bolic representations using its concept of resonance. Individual features are meaningless, just like pixels in a picture are meaningless. A learned category, or symbol, is sensitive to the global patterning of features, but cannot represent the "contents" of the experience, including their conscious qualia, due to the very fact that a categ01y is a compressed, or symbolic, representation. Resonance between these two types of information converts the pattern of attended features into a coherent contextsensitive state that is linked to its symbol through feedback. This coherent state, which binds distributed features and symbolic categories, can enter consciousness. ART predicts that all conscious states are resonant states. In particular, resonance binds spatially distributed features into a synchronous equilibrium or oscillation. Such synchronous states attracted interest after being reported in neurophysiological experiments. They were predicted in the 1970's when ART was introduced (see Grossberg, 1999b). Recent neurophysiological experiments have supported other ART predictions (Engel eta!., 2001; Pollen, 1999; Raizada and Grossberg, 2003). Fuzzy ART learns explicitly decodable Fuzzy IF-THEN rules (Carpenter et a!., 1992). Thus ART accommodates symbols and rules, as well as sub-symbolic distributed computations. Natural Language: ART has not yet modeled language. Rather, it is filling a gap that ACT-R has left open: "ACT-R lacks any theory of ... speech perception or speech production" (p. 22). ART is clarifying the perceptual units of speech perception, word recognition, working mem01y, and sequential planning chunks on which the brain builds language (e.g., Boardman et a!., 1999; Bradski eta!., 1994; Grossberg, 1978a, 1978b, 1999b; Grossberg eta!., 1997; Grossberg and Myers, 2000; Grossberg and Stone, 1986a, 1986b). Such studies suggest that a radical rethinking of psychological space and time is needed to understand language, and to accommodate such radical claims as: Conscious speech is a resonant wave. ACT -R also does not have 'mechanisms ... [of] perceptual recognition, mental imagery, emotion, and motivation" (p. 22). These are all areas where ART has detailed models (e.g., Grossberg, 2000a, 2000c). Speech production uses complementmy V AM-like mechanisms (Callan et al., 2000; Guenther, 1995). After perceptual units in vision became sufficiently clear, rapid progress ensued at all levels of vision (http://www.cns.bu.edu/Profiles/Grossberg). This should also happen for language. Development: ART predicted since 1976 that processes of cortical development in the infant are on a continuum with processes of lem·ning in the adult, a prediction increasing supported recently (e.g., Kandel and O'Dell, 1992). Evolution: "Cognitive plasticity ... What enables this plasticity in the architecture?" (p. 29). ART clarifies how the ability to learn quickly and stably throughout life implies cognitive properties like intention, attention, hypothesis testing, and resonance. Although Bayesian properties emerge from ART circuits, ART deals with novel experiences where no priors are defined. Brain: CM connectionism is said to be Best, although its main algorithms are biologically unrealizable. ART and YAM are realized in verified brain circuits. It might be prudent to include more ART in ACT. I also recommend eliminating straw man "debates" that do not reflect the true state of knowledge in cognitive science. REFERENCES Boardman, I., Grossberg, S., Myers, C., and Cohen, M. (1999). Neural dynamics of perceptual order and context effects for variable-rate speech syllables. Perception & Psychophysics, 61, 1477-1500. Brown, J., Bullock, D., and Grossberg, S. (1999) How the basal ganglia use parallel excitatory and inhibitory learning pathways to selectively respond to unexpected rewarding cues. Journal of Neuroscience, 19, 10502-10511. Bradski, G., Carpenter, G.A., and Grossberg, S. (1994). STORE working memory networks for storage and recall of arbitrary temporal sequences. Biological Cybernetics, 71, 469-480. Bullock, D., Grossberg, S., and Guenther, F. (1993). A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. Journal of Cognitive Neuroscience, 5, 408435. Bullock, D., Grossberg, S., and Mannes, C. (1993). A neural network model for cursive script production. Biological Cybernetics, 70, 15-28. Callan, D.E., Kent, R.D., Guenther, F.H., and Vorperian, H.K. (2000). An auditmy-feedbackbased neural network model of speech production that is robust to developmental changes in the size and shape of the articulatory system. Journal of Speech, Language, and Hearing Research, 43,721-736. Carpenter, G.A., Gopal, S., Macomber, S., Martens, S., Woodcock, C.E., and Franklin, J. (1999). A neural network method for efficient vegetation mapping. Remote Sensing of Environment, 70, 326-338. Carpenter, G.A. and Grossberg, S. (Eds.) (1991). Pattern Recognition by Self-Organizing Neural Networks. Cambridge, MA: MIT Press, 1991. Carpenter, G.A., Grossberg, S., Markuzon, N, Reynolds, J.H., and Rosen, D. (1992). Fuzz