Read Montaguel and The Predictive Brain : Temporal

Introduction Some forms of synaptic plasticity depend o n the temporal coincidence of presynaptic activity and postsynaptic response. This requirement is consistent with the Hebbian, o r correlational, type of learning rule used in many neural network models. Recent evidence suggests that synaptic plasticity may depend in part o n the production of a membrane permeant-diffusible signal so that spatial volume may also be involved in correlational learning rules. This latter form of synaptic change has been called volume learning. In both Hebbian and volume learning rules, interaction among synaptic inputs depends o n the degree of coincidence of the inputs and is otherwise insensitive to their exact temporal order. Conditioning experiments and psychophysical studies have shown, however, that most animals are highly sensitive to the temporal order of the sensory inputs. Although these experiments assay the behavior of the entire animal o r perceptual system, they raise the possibility that nervous systems may be sensitive to temporally ordered events at many spatial and temporal scales. We suggest here the existence of a new class of learning rule, called apredictiue Hebbian learning rule, that is sensitive to the temporal ordering of synaptic inputs. We show how this predictive learning rule could act at single synaptic connections and through diffuse neuromodulatory systems. Most biologically feasible theories of how experience-dependent changes take place in real neuronal networks use some variant of the notion that the efficacy or "strength of a synaptic connection from one cell to another can be modified on the basis of its history. In this theoretical work it is generally assumed that modifications of synaptic eEcacy, by acting over a large population of synapses, can account for interesting forms of learning and memory. This theoretical assumption prevails primarily because of its intuitive appeal, its accessibility to analysis, some provocative relations to biological data, and a lack of good alternatives. Recent work demonstrates that simple abstract learning algorithms, if given appropriately coded input, can produce complicated mappings from input to output. These efforts include networks that learn to pronounce written text (Sejnowski and Rosenberg 1987), play master level backgammon (Tesauro 1994), and recognize handwritten characters (Le Cun et al. 1990). As pointed out by Crick (1989) and others, many of these efforts are not good models of the vertebrate brain; however, they can be quite valuable for identrfying the informational requirements LEARNING & MEMORY 1 :1-33 O 1994 by Cold Spring Harbor Laboratory Press ISSN1072-0502194 $5.00 L E A R N I N G M E M O R Y Montague and Sejnowski involved in specific tasks. Moreover, they point out some of the computational constraints to which brains are subject. An awareness of the computational constraints involved in a particular problem can guide theories that explain how real brains are constructed (Churchlanci and Sejnowski 1992). Although abstract networks have provided some insight into the top-down constraints that nervous systems face, these approaches are of limited use in gaining insight into how various problems have been solved by real brains. For example, the actual learning mechanisms that are used in biological systems also satisfy additional constraints that arise from the known properties of neurons and synapses. In this paper we focus on learning rules that are supported by biological data and consider the strengths and weaknesses of these rules by measuring them against both computational and biological constraints. Taking this dual approach, we show that computational concerns applicable to the behavior and survival of the animal can work hand in hand with biologically feasible synaptic mechanisms to explain and predict experimental data. Correlational Theoretical accounts of how neural activity actually changes synaptic Learning function typically rely on a local correlational learning rule to model Rules-Learning synaptic plasticity. A correlational learning rule, often called a Hebbian Driven by Temporal learning rule, uses the correlation between presynaptic activity and Coincidence postsynaptic response to drive changes in synaptic efficacy (Fig. 1) (Hertz et al. 1991; Churchland and Sejnowski 1992). One simple expression of a Hebbian learning rule is where, at time t, w(t) is a connection strength (weight), x ( t ) is a measure of presynaptic activity, y(t) is a measure of postsynaptic activity (e.g., firing rate or probability of firing), and q is a fuced learning rate. This kind of learning rule is called local because the signals sufficient for changing synaptic eff~cacy are assumed to be generated locally at each synaptic contact. One form of this learning rule was initially proposed by Donald Hebb in 1949 (Hebb 1949). Subsequent theoretical and computational efforts have exploited Hebb's idea and used correlational learning rules to account successfully for aspects of map formation and self-organization of visual and somatosensory cortex (von der Malsburg 1973; von der Malsburg and Willshaw 1977; Montague et al. 199 1). For example, various computational schemes employing Hebbian learning rules have accounted for the formation of cortical receptive fields (Bienenstock et al. 1982; Linsker 1986, 1988), ocular dominance columns (Miller et al. 1989), orientation maps (von der Malsburg 1973; Obermayer et al. 1990; Miller 1994), dii-ectional selectivity (Sereno and Sereno 1991), and disparity tuning (Berns et al. 1993). Correlational learning rules also provide a reasonable theoretical framework for synaptic plasticity observed in the hippocampus (Kelso et al. 1986; Bliss and Lynch 1988), cerebellum (Ito 1986, 1989), and neocortex (Kirkwood et al. 1993). Below, we review some of the biological evidence from the vertebrate nervous system that supports this simple learning rule as a descriptor of synaptic change during both activity-dependent development and synaptic modification in the adult. We subsequently suggest that changes L E A R N I N G M E M O R Y THE PREDICTIVE BRAIN Figure 1: Hebbian Learning. (A) Inputs xi provide excitatory drive to a neuron through connection strengths or "weights". Inputs x, and x, are sufficiently correlated to permit cooperation along a section of dendrite (shaded area) through voltage or second messengers. Through an expression like equation 1, the weights of these connections will be increased. Input x, is not active during this coincident activation of x, and x,. The weight of x,'s connection could be decreased by a depression rule that depressed all synaptic contacts that were not sufficiently correlated with the postsynaptic response (shaded area). Without such a rule, weights can grow without bound. To prevent this, a homeostatic constraint that limits the total synaptic strength supported by the recipient neuron is typically used. This i s just one possible way to normalize the weights. The issue of how and why normalization is biologically reasonable is critical.' Normalization can give stability to the Hebb rule, but, depending on its implementation, it can cause the weight vector to converge to different values. In the presence of additional constraints (see text) for the learning rule, a Hebb rule will extract the principal component from the correlations in the input patterns that occur and the vector of weights wil l come to point in the direction of the first principle component of the "data" generated by the input activities (Oja 1982). The pattern of weights that develops can be analyzed in terms of the covariance matrix of the input activities (see text). (B ) Graph of input activity along two inputs, x, and x, . Each point i s a pair of activity levels for the two inputs in (A). The inputs cluster along a straight line, indicating a strong correlation. The approximate direction of the principal component i s along this line. in this description are required by both experimental and theoretical work. The primary change is predicated on the need for brain mechanisms sensitive to temporally ordered input, a problem that has most likely been solved by brains across a range of spatiotemporal scales. We marshal arguments and review detailed evidence in support of this suggestion and point out those aspects of the proposed changes that are important for understanding learning and memory in the vertebrate brain. Developmental In the vertebrate nervous system, afferent axons find their appropriate Evidence for Hebbian target structures through interactions with local environmental cues and Learning Rules target-derived information (Bonhoeffer and Huff 1985; Dodd and Jesse1 1988; Stuermer 1988; Harris 1989; Heffner et al. 1990; O'Leary et al. 1990; Placzek et al. 1990; Stretevan 1990). After reaching target structures, there is strong evidence that activity-dependent processes are critical in determining the development of mappings between peripheral sensory structures and their more centrally located target structures, including the optic tectum, thalamus, and cerebral cortex (Hubel and Wiesel 1965, 1970; Hubel et al. 1977; Meyer 1982; Stryker and Harris 1986; Stretevan et al. 1988). Specific mappings arise in these targets because temporal contiguity in axonal firing is somehow translated into L E A R N I N G M E M O R Y Montague and Sejnowski spatial contiguity of synaptic contacts. Hence, activity-dependent processes are involved at least with the initial self-otganizalion of mappings in the tectum, thalamus, and cortex. After normal developmental periods, activiw-dependent processes are also involved in the reorganization of sensory mappings in the adult. For example, the adult cerebral cortex has been shown to be surprisingly plastic (for review, see Kaas 1991; Merzenich and Sameshima 1993) following changes to the environment such as retinal damage (Kaas et al. 1990; Gilbert and W