Critical evidence for the prediction error theory in associative learning

In associative learning in mammals, it is widely accepted that the discrepancy, or error, between actual and predicted reward determines whether learning occurs. Complete evidence for the prediction error theory, however, has not been obtained in any learning systems: Prediction error theory stems from the finding of a blocking phenomenon, but blocking can also be accounted for by other theories, such as the attentional theory. We demonstrated blocking in classical conditioning in crickets and obtained evidence to reject the attentional theory. To obtain further evidence supporting the prediction error theory and rejecting alternative theories, we constructed a neural model to match the prediction error theory, by modifying our previous model of learning in crickets, and we tested a prediction from the model: the model predicts that pharmacological intervention of octopaminergic transmission during appetitive conditioning impairs learning but not formation of reward prediction itself, and it thus predicts no learning in subsequent training. We observed such an “auto-blocking”, which could be accounted for by the prediction error theory but not by other competitive theories to account for blocking. This study unambiguously demonstrates validity of the prediction error theory in associative learning.

[1]  Pranay Goel,et al.  A neuronal network for the logic of Limax learning , 2006, Journal of Computational Neuroscience.

[2]  Yukihisa Matsumoto,et al.  Analysis and modeling of neural processes underlying sensory preconditioning , 2013, Neurobiology of Learning and Memory.

[3]  M. Gewecke,et al.  Epinastine, a highly specific antagonist of insect neuronal octopamine receptors. , 1998, European journal of pharmacology.

[4]  B. Webb,et al.  Elemental and non-elemental olfactory learning in Drosophila , 2011, Neurobiology of Learning and Memory.

[5]  R. R. Miller,et al.  Assessment of the Rescorla-Wagner model. , 1995, Psychological bulletin.

[6]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[7]  M. Bitterman,et al.  Intermodal blocking in honeybees , 2001, The Quarterly journal of experimental psychology. B, Comparative and physiological psychology.

[8]  M. Bitterman,et al.  Intramodal blocking in honeybees , 1997 .

[9]  Holger Kessler,et al.  Maps to models , 2004 .

[10]  B. H. Smith,et al.  The olfactory memory of the honeybee Apis mellifera. II. Blocking between odorants in binary mixtures. , 1994, The Journal of experimental biology.

[11]  M. Mizunami,et al.  Chapter 41 - Olfactory and Visual Learning in Cockroaches and Crickets , 2013 .

[12]  Terry Crow,et al.  Pavlovian conditioning of Hermissenda: current cellular, molecular, and circuit perspectives. , 2004, Learning & memory.

[13]  M. Mizunami,et al.  Roles of octopaminergic and dopaminergic neurons in appetitive and aversive memory recall in an insect , 2009, BMC Biology.

[14]  M. Mizunami,et al.  Temporal determinants of long-term retention of olfactory memory in the cricket Gryllus bimaculatus. , 2002, The Journal of experimental biology.

[15]  M. Heisenberg Mushroom body memoir: from maps to models , 2003, Nature Reviews Neuroscience.

[16]  N. Mackintosh A Theory of Attention: Variations in the Associability of Stimuli with Reinforcement , 1975 .

[17]  M. Mizunami,et al.  Why the carrot is more effective than the stick: Different dynamics of punishment memory and reward memory and its possible biological basis , 2009, Neurobiology of Learning and Memory.

[18]  Ronald L. Davis,et al.  Traces of Drosophila Memory , 2011, Neuron.

[19]  W. Schultz Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[20]  M. Bitterman,et al.  Within-subjects experiments on blocking and facilitation in honeybees (Apis mellifera). , 2008, Journal of comparative psychology.

[21]  B. H. Smith,et al.  Blocking and the detection of odor components in blends. , 2000, The Journal of experimental biology.

[22]  M. Mizunami,et al.  Participation of octopaminergic reward system and dopaminergic punishment system in insect olfactory learning revealed by pharmacological study , 2005, The European journal of neuroscience.

[23]  Randolf Menzel,et al.  Small Brains, Bright Minds , 2006, Cell.

[24]  G. Rubin,et al.  A subset of dopamine neurons signals reward for odour memory in Drosophila , 2012, Nature.

[25]  J. Prados,et al.  Blocking in rats, humans and snails using a within-subjects design , 2013, Behavioural Processes.

[26]  Ralph R. Miller,et al.  The comparator hypothesis: A response rule for the expression of associations. , 1988 .

[27]  E. Kandel The Molecular Biology of Memory Storage: A Dialogue Between Genes and Synapses , 2001, Science.

[28]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[29]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[30]  L. Matzel,et al.  Higher-order associative processing in Hermissenda suggests multiple sites of neuronal modulation. , 1996, Learning & memory.

[31]  B. H. Smith,et al.  The olfactory memory of the honeybee Apis mellifera. III. Bilateral sensory input is necessary for induction and expression of olfactory blocking. , 1997, The Journal of experimental biology.

[32]  M. Bitterman,et al.  Blocking and pseudoblocking: New control experiments with honeybees , 2006, Quarterly journal of experimental psychology.

[33]  H. Lachnit,et al.  Olfactory blocking and odorant similarity in the honeybee. , 2005, Learning & memory.

[34]  A. Gelperin,et al.  An analysis of associative learning in a terrestrial mollusc , 1981, Journal of comparative physiology.

[35]  H. Siegelmann,et al.  ASSOCIATIVE LEARNING , 2017 .

[36]  B. Brembs,et al.  Conditioning with compound stimuli in Drosophila melanogaster in the flight simulator. , 2001, The Journal of experimental biology.

[37]  F. Acebes,et al.  Blocking of conditioning of tentacle lowering in the snail (Helix aspersa) , 2009, Quarterly journal of experimental psychology.

[38]  M. Mizunami,et al.  Roles of octopaminergic and dopaminergic neurons in mediating reward and punishment signals in insect visual learning , 2006, The European journal of neuroscience.

[39]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.