Reconciling the influence of predictiveness and uncertainty on stimulus salience: a model of attention in associative learning

Theories of selective attention in associative learning posit that the salience of a cue will be high if the cue is the best available predictor of reinforcement (high predictiveness). In contrast, a different class of attentional theory stipulates that the salience of a cue will be high if the cue is an inaccurate predictor of reinforcement (high uncertainty). Evidence in support of these seemingly contradictory propositions has led to: (i) the development of hybrid attentional models that assume the coexistence of separate, predictiveness-driven and uncertainty-driven mechanisms of changes in cue salience; and (ii) a surge of interest in identifying the neural circuits underpinning these mechanisms. Here, we put forward a formal attentional model of learning that reconciles the roles of predictiveness and uncertainty in salience modification. The issues discussed are relevant to psychologists, behavioural neuroscientists and neuroeconomists investigating the roles of predictiveness and uncertainty in behaviour.

[1]  D. Whitteridge.  Lectures on Conditioned Reflexes , 1942, Nature.

[2]  R. F. Thompson,et al.  Habituation: a model phenomenon for the study of neuronal substrates of behavior. , 1966, Psychological review.

[3]  John Garcia,et al.  Relation of cue to consequence in avoidance learning , 1966 .

[4]  L. Kamin.  Attention-like processes in classical conditioning , 1967 .

[5]  J. Konorski.  Integrative activity of the brain , 1967 .

[6]  N. Mackintosh,et al.  Mechanisms of animal discrimination learning , 1971 .

[7]  P. Rozin,et al.  Specific hungers and poison avoidance as adaptive specializations of learning. , 1971, Psychological review.

[8]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[9]  R. Lubow.  Latent inhibition. , 1973, Psychological bulletin.

[10]  M. J. Scavio.  Classical-classical transfer: effects of prior aversive conditioning upon appetitive conditioning in rabbits (Oryctolagus cuniculus). , 1974, Journal of comparative and physiological psychology.

[11]  N. Mackintosh.  A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[12]  N. Mackintosh,et al.  Surprise and the attenuation of blocking. , 1976 .

[13]  R. Rescorla,et al.  Stimulus similarity as a determinant of Pavlovian conditioning. , 1977, Journal of experimental psychology. Animal behavior processes.

[14]  J. Pearce,et al.  Inhibitory interactions between appetitive and aversive stimuli. , 1977 .

[15]  J. Pearce,et al.  Inhibitory interactions between appetitive and aversive stimuli. , 1977 .

[16]  P. Holland.  Conditioned stimulus as a determinant of the form of the Pavlovian conditioned response. , 1977, Journal of experimental psychology. Animal behavior processes.

[17]  N. Mackintosh,et al.  Reinforcer specificity in the enhancement of conditioning by posttrial surprise. , 1979 .

[18]  J. Pearce,et al.  Latent inhibition of a CS during CS-US pairings. , 1979, Journal of experimental psychology. Animal behavior processes.

[19]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[20]  W. F. Hill,et al.  Mechanisms of Learning and Motivation: A Memorial Volume to Jerzy Konorski , 1980 .

[21]  N. Donegan.  Priming-produced facilitation or diminution of responding to a Pavlovian unconditioned stimulus. , 1981, Journal of experimental psychology. Animal behavior processes.

[22]  A. Dickinson.  Contemporary Animal Learning Theory , 1981 .

[23]  J. Pearce,et al.  Restoring the Associability of a Pre-Exposed CS by a Surprising Event , 1982 .

[24]  J. Pearce,et al.  The Influence of Partial Reinforcement on Serial Autoshaping with Pigeons , 1983, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[25]  J. Pearce,et al.  The strength of the orienting response during Pavlovian conditioning. , 1984, Journal of experimental psychology. Animal behavior processes.

[26]  N. Mackintosh,et al.  Context specificity of conditioning, extinction, and latent inhibition. , 1984 .

[27]  Geoffrey Hall,et al.  Differential effects of contextual change on latent inhibition and on the habituation of an orienting response , 1985 .

[28]  T. Devietti,et al.  Latent inhibition: A trace conditioning phenomenon? , 1987 .

[29]  N. Mackintosh,et al.  Context Specificity of Conditioning and Latent Inhibition: Evidence for a Dissociation of Latent Inhibition and Associative Interference , 1987 .

[30]  R. C. Honey,et al.  Overshadowing and Blocking Procedures in Latent Inhibition , 1988, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[31]  J. Pearce,et al.  The orienting response as an index of stimulus associability in rats. , 1988, Journal of experimental psychology. Animal behavior processes.

[32]  J. Pearce,et al.  Restoration of the Orienting Response to a Light by a Change in its Predictive Accuracy , 1992 .

[33]  J. Pearce.  Similarity and discrimination: a selective review and a connectionist model. , 1994, Psychological review.

[34]  N. Mackintosh,et al.  Latent Inhibition, Context Specificity, and Context Familiarity , 1994, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[35]  N. Schmajuk,et al.  Latent inhibition: a neural network approach. , 1996, Journal of experimental psychology. Animal behavior processes.

[36]  M. Papini,et al.  Consequences of Surprising Reward Omissions , 1997 .

[37]  J. Pearce,et al.  The role of attention in the solution of conditional discriminations , 1998 .

[38]  J. Pearce,et al.  Acquired distinctiveness is controlled by stimulus relevance not correlation with reward , 1999 .

[39]  J. Kruschke.  Toward a unified model of attention in associative learning , 2001 .

[40]  R. Rescorla.  Comparison of the rates of associative change during acquisition and extinction. , 2002, Journal of experimental psychology. Animal behavior processes.

[41]  I. Mclaren,et al.  Learned Associability and Associative Change in Human Causal Learning , 2003, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[42]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[43]  Edgar H Vogel,et al.  Stimulus representation in SOP: II. An application to inhibition of delay , 2003, Behavioural Processes.

[44]  M. Pelley.  The Role of Associative History in Models of Associative Learning: A Selective Review and a Hybrid Model: , 2004 .

[45]  I. Mclaren,et al.  The outcome specificity of learned predictiveness effects: parallels between human causal learning and animal conditioning. , 2005, Journal of experimental psychology. Animal behavior processes.

[46]  J. O'Doherty,et al.  Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain , 2006, PLoS biology.

[47]  S. Quartz,et al.  Neural Differentiation of Expected Reward and Risk in Human Subcortical Structures , 2006, Neuron.

[48]  J. O'Doherty,et al.  Reward Value Coding Distinct From Risk Attitude-Related Uncertainty Coding in Human Reward Systems , 2006, Journal of neurophysiology.

[49]  M. L. Le Pelley,et al.  Blocking of Human Causal Learning Involves Learned Changes in Stimulus Processing , 2007, Quarterly journal of experimental psychology.

[50]  Andy J. Wills,et al.  Predictive Learning, Prediction Errors, and Attention: Evidence from Event-related Potentials and Eye Tracking , 2007, Journal of Cognitive Neuroscience.

[51]  S. Killcross,et al.  Optional-shift behaviour in rats: A novel procedure for assessing attentional processes in discrimination learning , 2007, Quarterly journal of experimental psychology.

[52]  A. Dickinson,et al.  Attention and expectation in human predictive learning: The role of uncertainty , 2008, Quarterly journal of experimental psychology.

[53]  Colin Camerer,et al.  Explicit neural signals reflecting reward uncertainty , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[54]  Christopher J. Mitchell,et al.  Selective attention in human associative learning and recognition memory. , 2008, Journal of experimental psychology. General.

[55]  Guillem R. Esber,et al.  The nature of discrimination learning in pigeons , 2008, Learning & behavior.

[56]  M. Pelley,et al.  Disentangling the attentional deficit in schizophrenia: Pointers from schizotypy , 2010, Psychiatry Research.

[57]  Christopher J. Mitchell,et al.  Attention and Associative Learning: From Brain to Behaviour , 2010 .

[58]  Ethan S. Bromberg-Martin,et al.  Dopamine in Motivational Control: Rewarding, Aversive, and Alerting , 2010, Neuron.

[59]  Peter M. Jones,et al.  Two kinds of attention in Pavlovian conditioning: evidence for a hybrid model of learning. , 2010, Journal of experimental psychology. Animal behavior processes.

[60]  N. Mackintosh,et al.  Two theories of attention: a review and a possible integration , 2010 .

[61]  Computational Models of Conditioning: The hybrid modeling approach to conditioning , 2010 .

[62]  Tom Beesley,et al.  Stereotype formation: biased by association. , 2010, Journal of experimental psychology. General.

[63]  M. Pelley,et al.  Learned predictiveness effects following single-cue training in humans , 2010, Learning & behavior.