Interactions of timing and prediction error learning

[1]  L. Kamin.  Predictability, surprise, attention, and conditioning , 1967 .

[2]  L. Kamin.  Attention-like processes in classical conditioning , 1967 .

[3]  B. Campbell,et al.  Punishment and aversive behavior , 1969 .

[4]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[5]  J. Ayres,et al.  The Kamin blocking effect with variable-duration CSs , 1979 .

[6]  P. Frey,et al.  Blocking in eyelid conditioning: Effect of changing the CS-US interval and introducing an intertriai stimulus , 1979 .

[7]  H. Terrace,et al.  Autoshaping and Conditioning Theory , 1980 .

[8]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[9]  S. Roberts,et al.  Isolation of an internal clock. , 1981, Journal of experimental psychology. Animal behavior processes.

[10]  E. Kehoe,et al.  Blocking acquisition of the rabbit's nictitating membrane response to serial conditioned stimuli , 1981 .

[11]  Schreurs Bg,et al.  The effects of changes in the CS-US interval during compound conditioning upon an other wise blocked element , 1982 .

[12]  J. Ayres,et al.  Blocking with serial and simultaneous compounds in a trace conditioning procedure , 1982 .

[13]  R. A. Hancock.  Tests of the conditioned reinforcement value of sequential stimuli in pigeons , 1982 .

[14]  S. Gaioni.  Blocking and nonsimultaneous compounds: Comparison of responding during compound conditioning and testing , 1982, The Pavlovian journal of biological science.

[15]  E. Kehoe.  CS-US contiguity and CS intensity in conditioning of the rabbit's nictitating membrane response to serial compound stimuli. , 1983, Journal of experimental psychology. Animal behavior processes.

[16]  J. Krebs,et al.  Time Horizons of Foraging Animals a , 1984, Annals of the New York Academy of Sciences.

[17]  J. Gibbon,et al.  Annals of the New York Academy of Sciences. Volume 423. Timing and Time Perception Held at New York on 10-13 May 1983, , 1984 .

[18]  R M Church,et al.  Scalar Timing in Memory , 1984, Annals of the New York Academy of Sciences.

[19]  W. Meck.  Attentional Bias between Modalities: Effect on the Internal Clock, Memory, and Decision Stages Used in Animal Time Discrimination a , 1984, Annals of the New York Academy of Sciences.

[20]  E. Kehoe,et al.  Temporal primacy overrides prior training in serial compound conditioning of the rabbit’s nictitating membrane response , 1987 .

[21]  P. Killeen,et al.  A behavioral theory of timing , 1988 .

[22]  Stephen Grossberg,et al.  Neural dynamics of adaptive timing and temporal discrimination during associative learning , 1989, Neural Networks.

[23]  Christopher Miall,et al.  The Storage of Time Intervals Using Oscillating Neurons , 1989, Neural Computation.

[24]  M. Davis,et al.  Temporal specificity of fear conditioning: effects of different conditioned stimulus-unconditioned stimulus intervals on the fear-potentiated startle effect. , 1989, Journal of experimental psychology. Animal behavior processes.

[25]  R. Church,et al.  Alternative representations of time, number, and rate , 1990, Cognition.

[26]  Richard S. Sutton,et al.  Time-Derivative Models of Pavlovian Reinforcement , 1990 .

[27]  R. R. Miller,et al.  Temporal encoding as a determinant of blocking. , 1993, Journal of experimental psychology. Animal behavior processes.

[28]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  John E. R. Staddon,et al.  MULTIPLE TIME SCALES IN SIMPLE HABITUATION , 1996 .

[30]  W. Meck.  Neuropharmacology of timing and time perception. , 1996, Brain research. Cognitive brain research.

[31]  T. Robbins,et al.  Neurobehavioural mechanisms of reward and motivation , 1996, Current Opinion in Neurobiology.

[32]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[33]  Armando Machado,et al.  Learning the temporal dynamics of behavior. , 1997 .

[34]  W. Schultz.  Predictive reward signal of dopamine neurons. , 1998, Journal of neurophysiology.

[35]  Kimberly S. Kirkpatrick,et al.  Are separate theories of conditioning and timing necessary? , 1998, Behavioural Processes.

[36]  R. Church,et al.  Temporal search as a function of the variability of interfood intervals. , 1998, Journal of experimental psychology. Animal behavior processes.

[37]  K. Lattal,et al.  Trial and intertrial durations in Pavlovian conditioning: issues of learning and performance. , 1999, Journal of experimental psychology. Animal behavior processes.

[38]  S. Klein,et al.  Handbook of contemporary learning theories , 2000 .

[39]  J. Nevin,et al.  Response strength and temporal control in fixed-interval schedules , 2000 .

[40]  P. Holland.  Trial and intertrial durations in appetitive conditioning in rats , 2000 .

[41]  C. Gallistel,et al.  Time, rate, and conditioning. , 2000, Psychological review.

[42]  Kimberly S. Kirkpatrick,et al.  Independent effects of stimulus and cycle duration in conditioning: The role of timing processes , 2000 .

[43]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[44]  T. Ono,et al.  Retrospective and prospective coding for predicted reward in the sensory thalamus , 2001, Nature.

[45]  G. Hall,et al.  Lesions of the Basolateral Amygdala Disrupt Selective Aspects of Reinforcer Representation in Rats , 2001, The Journal of Neuroscience.

[46]  M. Mauk,et al.  Latent Acquisition of Timed Responses in Cerebellar Cortex , 2001, The Journal of Neuroscience.

[47]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[48]  Stephen M. Rao,et al.  The evolution of brain activation during temporal processing , 2001, Nature Neuroscience.

[49]  E. Murray,et al.  The amygdala and reward , 2002, Nature Reviews Neuroscience.

[50]  Alex Kacelnik,et al.  Timing and Foraging: Gibbon's Scalar Expectancy Theory and Optimal Patch Exploitation , 2002 .

[51]  J. Richards,et al.  Effects of reinforcer magnitude on responding under differential-reinforcement-of-low-rate schedules of rats and pigeons. , 2002, Journal of the experimental analysis of behavior.

[52]  Christian Gaser,et al.  Processing of temporal information and the basal ganglia: new evidence from fMRI , 2003, Experimental Brain Research.

[53]  Kimberly S. Kirkpatrick,et al.  Packet theory of conditioning and timing , 2002, Behavioural Processes.

[54]  P. Balsam,et al.  Timing at the Start of Associative Learning , 2002 .

[55]  C. Gallistel,et al.  Temporal landmarks: proximity prevails , 2003, Animal Cognition.

[56]  G. Schoenbaum,et al.  Encoding Predicted Outcome and Acquired Value in Orbitofrontal Cortex during Cue Sampling Depends upon Input from Basolateral Amygdala , 2003, Neuron.

[57]  A. Kelley,et al.  Nucleus accumbens opioid, GABaergic, and dopaminergic modulation of palatable food motivation: contrasting effects revealed by a progressive ratio study in the rat. , 2003, Behavioral neuroscience.

[58]  J. O'Doherty,et al.  Encoding Predictive Reward Value in Human Amygdala and Orbitofrontal Cortex , 2003, Science.

[59]  M. Nicolelis,et al.  Interval timing and the encoding of signal duration by ensembles of cortical and striatal neurons. , 2003, Behavioral neuroscience.

[60]  M. Shadlen,et al.  Representation of Time by Neurons in the Posterior Parietal Cortex of the Macaque , 2003, Neuron.

[61]  Kimberly S. Kirkpatrick,et al.  Tracking of the expected time to reinforcement in temporal conditioning procedures , 2003, Learning & behavior.

[62]  F. Vidal,et al.  Functional Anatomy of the Attentional Modulation of Time Estimation , 2004, Science.

[63]  W. Senn,et al.  Climbing Neuronal Activity as an Event-Based Cortical Representation of Time , 2004, The Journal of Neuroscience.

[64]  Samuel M. McClure,et al.  Separate Neural Systems Value Immediate and Delayed Monetary Rewards , 2004, Science.

[65]  E. Rolls,et al.  The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology , 2004, Progress in Neurobiology.

[66]  Kimberly S. Kirkpatrick,et al.  Temporal learning in random control procedures. , 2004, Journal of experimental psychology. Animal behavior processes.

[67]  W. Meck,et al.  Cortico-striatal circuits and interval timing: coincidence detection of oscillatory processes. , 2004, Brain research. Cognitive brain research.

[68]  P. Holland,et al.  Amygdala–frontal interactions and reward expectancy , 2004, Current Opinion in Neurobiology.

[69]  T. Robbins,et al.  Contrasting Roles of Basolateral Amygdala and Orbitofrontal Cortex in Impulsive Choice , 2004, The Journal of Neuroscience.

[70]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[71]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[72]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[73]  J. Feldon,et al.  Double dissociation of the effects of selective nucleus accumbens core and shell lesions on impulsive‐choice behaviour and salience learning in rats , 2005, The European journal of neuroscience.

[74]  Trevor W Robbins,et al.  Interactions between Serotonin and Dopamine in the Control of Impulsive Choice in Rats: Therapeutic Implications for Impulse Control Disorders , 2005, Neuropsychopharmacology.

[75]  P. Couvillon,et al.  Temporal control of conditioned responding in goldfish. , 2005, Journal of experimental psychology. Animal behavior processes.

[76]  W. Meck.  Neuroanatomical localization of an internal clock: A functional link between mesolimbic, nigrostriatal, and mesocortical dopaminergic systems , 2006, Brain Research.

[77]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[78]  A. Odum,et al.  Effects of prefeeding, intercomponent-interval food, and extinction on temporal discrimination and pacemaker rate , 2006, Behavioural Processes.

[79]  Kimberly S. Kirkpatrick,et al.  Interval duration effects on blocking in appetitive conditioning , 2006, Behavioural Processes.

[80]  M. Frank,et al.  Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. , 2006, Psychological review.

[81]  Yudhijit Bhattacharjee.  A Timely Debate About the Brain , 2006, Science.

[82]  W. Schultz.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[83]  JaneR . Taylor,et al.  ΔFosB in the Nucleus Accumbens Regulates Food-Reinforced Instrumental Behavior and Motivation , 2006, The Journal of Neuroscience.

[84]  Kimberly S. Kirkpatrick,et al.  Overshadowing and stimulus duration. , 2007, Journal of experimental psychology. Animal behavior processes.

[85]  J. Deakin,et al.  Effect of quinolinic acid-induced lesions of the nucleus accumbens core on performance on a progressive ratio schedule of reinforcement: implications for inter-temporal choice , 2008, Psychopharmacology.

[86]  C. M. Bradshaw,et al.  Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis , 2007, Psychopharmacology.

[87]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[88]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[89]  Elliot A. Ludvig,et al.  The effects of reinforcer magnitude on timing in rats. , 2007, Journal of the experimental analysis of behavior.

[90]  R. Church,et al.  A modular theory of learning and performance , 2007, Psychonomic bulletin & review.

[91]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[92]  Richard S. Sutton,et al.  Stimulus Representation and the Timing of Reward-Prediction Errors in Models of the Dopamine System , 2008, Neural Computation.

[93]  Franck Vidal,et al.  Timing, Storage, and Comparison of Stimulus Duration Engage Discrete Anatomical Components of a Perceptual Timing Network , 2008, Journal of Cognitive Neuroscience.

[94]  M. Nicolelis,et al.  Decoding of temporal intervals from cortical ensemble activity. , 2008, Journal of neurophysiology.

[95]  Warren H Meck,et al.  Cortico-striatal Representation of Time in Animals and Humans This Review Comes from a Themed Issue on Cognitive Neuroscience Edited Evidence from Patient Populations and Electrical Potentials Neuroimaging Evidence Using Fmri and Pet , 2022 .

[96]  A. Nobre,et al.  Dissociating explicit timing from temporal expectation with fMRI , 2008, Current Opinion in Neurobiology.

[97]  Ryan D Ward,et al.  Impaired timing precision produced by striatal D2 receptor overexpression is mediated by cognitive and motivational deficits. , 2009, Behavioral neuroscience.

[98]  C. Büchel,et al.  Overlapping and Distinct Neural Systems Code for Subjective Value during Intertemporal and Risky Decision Making , 2009, The Journal of Neuroscience.

[99]  Ryan D Ward,et al.  Timing and anticipation: conceptual and methodological approaches , 2009, The European journal of neuroscience.

[100]  J. Staddon,et al.  The behavioral economics of choice and interval timing. , 2009, Psychological review.

[101]  Benjamin Morillon,et al.  Three Stages and Four Neural Systems in Time Estimation , 2009, The Journal of Neuroscience.

[102]  C. Gallistel,et al.  Temporal maps and informativeness in associative learning , 2009, Trends in Neurosciences.

[103]  Y. Niv.  Reinforcement learning in the brain , 2009 .

[104]  T. Maia.  Reinforcement learning, conditioning, and the brain: Successes and challenges , 2009, Cognitive, affective & behavioral neuroscience.

[105]  P. Glimcher,et al.  The Neurobiology of Decision: Consensus and Controversy , 2009, Neuron.

[106]  W. Schultz,et al.  Short-Term Temporal Discounting of Reward Value in Human Ventral Striatum , 2009, Journal of neurophysiology.

[107]  Kimberly S. Kirkpatrick,et al.  Reward value effects on timing in the peak procedure , 2009 .

[108]  Eduardo Alonso,et al.  Computational Neuroscience for Advancing Artificial Intelligence: Models, Methods and Applications , 2010 .

[109]  W. Schultz,et al.  Coding of Reward Risk by Orbitofrontal Neurons Is Mostly Distinct from Coding of Reward Value , 2010, Neuron.

[110]  Kimberly S. Kirkpatrick,et al.  Reward magnitude effects on temporal discrimination. , 2010, Learning and motivation.

[111]  Ulrike Basten,et al.  How the brain integrates costs and bene fi ts during decision making , 2010 .

[112]  C. Büchel,et al.  Neural representations of subjective reward value , 2010, Behavioural Brain Research.

[113]  Within-session modulation of timed anticipatory responding: When to start responding , 2010, Behavioural Processes.

[114]  Eduardo Alonso,et al.  Temporal uncertainty during overshadowing: A temporal difference account , 2010 .

[115]  Fuat Balci,et al.  Motivational effects on interval timing in dopamine transporter (DAT) knockdown mice , 2010, Brain Research.

[116]  Thomas E. Hazy,et al.  Neural mechanisms of acquired phasic dopamine responses in learning , 2010, Neuroscience & Biobehavioral Reviews.

[117]  B. Balleine,et al.  At the limbic–motor interface: disconnection of basolateral amygdala from nucleus accumbens core and shell reveals dissociable components of incentive motivation , 2010, The European journal of neuroscience.

[118]  W. Roberts,et al.  The effects of cue competition on timing in pigeons , 2010, Behavioural Processes.

[119]  J. Deakin,et al.  Choice between reinforcer delays versus choice between reinforcer magnitudes: Differential Fos expression in the orbital prefrontal cortex and nucleus accumbens core , 2010, Behavioural Brain Research.

[120]  C. Gallistel,et al.  Time and Associative Learning. , 2010, Comparative cognition & behavior reviews.

[121]  Kimberly S. Kirkpatrick,et al.  The role of the nucleus accumbens core in impulsive choice, timing, and reward processing. , 2010, Behavioral neuroscience.

[122]  Jonathan D. Cohen,et al.  A Model of Interval Timing by Neural Integration , 2011, The Journal of Neuroscience.

[123]  Marc G. Bellemare,et al.  A primer on reinforcement learning in the brain : Psychological, computational, and neural perspectives , 2011 .

[124]  Jan Peters,et al.  The neural mechanisms of inter-temporal decision-making: understanding variability , 2011, Trends in Cognitive Sciences.

[125]  Yoshua Bengio,et al.  Adaptive Drift-Diffusion Process to Learn Time Intervals , 2011, 1103.2382.

[126]  W. Meck,et al.  Neuroanatomical and Neurochemical Substrates of Timing , 2011, Neuropsychopharmacology.

[127]  Marcia L. Spetch,et al.  Reward magnitude and timing in pigeons , 2011, Behavioural Processes.

[128]  Andrew T. Marshall,et al.  Motivation and timing: Clues for modeling the reward system , 2012, Behavioural Processes.

[129]  Elliot A. Ludvig,et al.  Evaluating the TD model of classical conditioning , 2012, Learning & behavior.

[130]  Andre Luzardo,et al.  An adaptive drift-diffusion model of interval timing dynamics , 2013, Behavioural Processes.

[131]  Bilgehan Çavdaroğlu,et al.  Epistasis effects of dopamine genes on interval timing and reward magnitude in humans , 2013, Neuropsychologia.

[132]  H. Terrace,et al.  Sources of Variance in an Information Processing Theory of Timing , 2014 .