Interval timing under a behavioral microscope: Dissociating motivational and timing processes in fixed-interval performance

The distribution of latencies and interresponse times (IRTs) of rats was compared between two fixed-interval (FI) schedules of food reinforcement (FI 30 s and FI 90 s), and between two levels of food deprivation. Computational modeling revealed that latencies and IRTs were well described by mixture probability distributions embodying two-state Markov chains. Analysis of these models revealed that only a subset of latencies is sensitive to the periodicity of reinforcement, and prefeeding only reduces the size of this subset. The distribution of IRTs suggests that behavior in FI schedules is organized in bouts that lengthen and ramp up in frequency with proximity to reinforcement. Prefeeding slowed down the lengthening of bouts and increased the time between bouts. When concatenated, latency and IRT models adequately reproduced sigmoidal FI response functions. These findings suggest that behavior in FI schedules fluctuates in and out of schedule control; an account of such fluctuation suggests that timing and motivation are dissociable components of FI performance. These mixture-distribution models also provide novel insights on the motivational, associative, and timing processes expressed in FI performance. These processes may be obscured, however, when performance in timing tasks is analyzed in terms of mean response rates.

[1]  Lewis A. Bizo,et al.  How reinforcement context affects temporal production and categorization , 1998 .

[2]  J. Myerson,et al.  The kinetics of choice: An operant systems analysis. , 1980 .

[3]  Catalin V. Buhusi,et al.  What makes us tick? Functional and neural mechanisms of interval timing , 2005, Nature Reviews Neuroscience.

[4]  M. Morgan Fixed Interval Schedules and Delay of Reinforcement , 1970 .

[5]  J. Staddon,et al.  Dynamics of waiting in pigeons. , 1996, Journal of the experimental analysis of behavior.

[6]  Yixin Fang,et al.  Asymptotic Equivalence between Cross-Validations and Akaike Information Criteria in Mixed-Effects Models , 2021 .

[7]  J. Staddon,et al.  Habituation, memory and the brain: the dynamics of interval timing , 2002, Behavioural Processes.

[8]  F. Balcı Interval Timing, Dopamine, and Motivation , 2014 .

[9]  Kimberly S. Kirkpatrick,et al.  Tracking of the expected time to reinforcement in temporal conditioning procedures , 2003, Learning & behavior.

[10]  Dénes Szűcs,et al.  Representational change and strategy use in children's number line estimation during the first years of primary school , 2012, Behavioral and Brain Functions.

[11]  T. Belke,et al.  Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement , 2006, Behavioural Processes.

[12]  A. Machado Learning the temporal dynamics of behavior. , 1997, Psychological review.

[13]  Lewis A. Bizo,et al.  Training with controlled reinforcer density: implications for models of timing , 1997 .

[14]  Federico Sanabria,et al.  The isolation of motivational, motoric, and schedule effects on operant performance: a modeling approach. , 2011, Journal of the experimental analysis of behavior.

[15]  S. Roberts,et al.  Variation of bar-press duration: Where do new responses come from? , 2006, Behavioural Processes.

[16]  B A Schneider,et al.  A two-state analysis of fixed-interval responding in the pigeon. , 1969, Journal of the experimental analysis of behavior.

[17]  H. Hurwitz Response-Duration of Lever Pressing in the Rat , 1954 .

[18]  J. Wearden,et al.  The Power Law and Weber's Law in Fixed-Interval Postreinforcement Pausing: A Scalar Timing Model , 1985 .

[19]  S. Roberts,et al.  Isolation of an internal clock. , 1981, Journal of experimental psychology. Animal behavior processes.

[20]  Wolfram Erlhagen,et al.  Learning to Time: a perspective. , 2009, Journal of the experimental analysis of behavior.

[21]  R. Ivry,et al.  Perception and production of temporal intervals across a range of durations: evidence for a common timing mechanism. , 1995, Journal of experimental psychology. Human perception and performance.

[22]  Gabriel J. Mazur,et al.  Validation of a method to assess ADHD-related impulsivity in animal models , 2015, Journal of Neuroscience Methods.

[23]  Federico Sanabria,et al.  Adaptation of timing behavior to a regular change in criterion , 2014, Behavioural Processes.

[24]  Sylvie Droit-Volet,et al.  The Effect of Feedback on Timing in Children and Adults: The Temporal Generalization Task , 2005, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[25]  A. Dagher,et al.  Dopamine Precursor Depletion Impairs Timing in Healthy Volunteers by Attenuating Activity in Putamen and Supplementary Motor Area , 2012, The Journal of Neuroscience.

[26]  Jeffrey N. Rouder,et al.  Default Bayes factors for ANOVA designs , 2012 .

[27]  R. Shull,et al.  Bouts of responding on variable-interval schedules: effects of deprivation level. , 2004, Journal of the experimental analysis of behavior.

[28]  J. Staddon,et al.  Time and memory: towards a pacemaker-free theory of interval timing. , 1999, Journal of the experimental analysis of behavior.

[29]  Jonathan D. Cohen,et al.  A Model of Interval Timing by Neural Integration , 2011, The Journal of Neuroscience.

[30]  C. Buhusi,et al.  Modeling Pharmacological Clock and Memory Patterns of Interval Timing in a Striatal Beat-Frequency Model with Realistic, Noisy Neurons , 2011, Front. Integr. Neurosci..

[31]  Edgar H Vogel,et al.  Stimulus representation in SOP: II. An application to inhibition of delay , 2003, Behavioural Processes.

[32]  J. Wearden,et al.  The comparative psychology of fixed-interval responding: Some quantitative analyses ☆ , 1991 .

[33]  R. Shull,et al.  Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules. , 1972, Journal of the experimental analysis of behavior.

[34]  Raymond C Pitts,et al.  Concurrent performance as bouts of behavior. , 2014, Journal of the experimental analysis of behavior.

[35]  J. Richards,et al.  DRL interresponse-time distributions: quantification by peak deviation analysis. , 1993, Journal of the experimental analysis of behavior.

[36]  Russell M. Church,et al.  A Turing test of a timing theory , 2005, Behavioural Processes.

[37]  M. Zeiler,et al.  Temporal control in fixed-interval schedules. , 1994, Journal of the experimental analysis of behavior.

[38]  C. Wynne,et al.  Effects of d-amphetamine on the behavior of pigeons exposed to the peak procedure , 2006, Behavioural Processes.

[39]  Marcia L. Spetch,et al.  Reward magnitude and timing in pigeons , 2011, Behavioural Processes.

[40]  Federico Sanabria,et al.  A microstructural analysis of schedule-induced polydipsia reveals incentive-induced hyperactivity in an animal model of ADHD , 2015, Behavioural Brain Research.

[41]  C. Plowright,et al.  Time estimation by pigeons on a fixed interval: the effect of pre-feeding , 2000, Behavioural Processes.

[42]  R. Church,et al.  Dynamics of temporal discrimination , 2005, Learning & behavior.

[43]  P. Killeen,et al.  Evidence for impulsivity in the Spontaneously Hypertensive Rat drawn from complementary response-withholding tasks , 2008, Behavioral and Brain Functions.

[44]  Ralph R. Miller,et al.  Timing: An attribute of associative learning , 2014, Behavioural Processes.

[45]  Justin A. Harris Changes in the distribution of response rates across the CS-US interval: Evidence that responding switches between two distinct states. , 2015, Journal of experimental psychology. Animal learning and cognition.

[46]  K. Kirkpatrick Interactions of timing and prediction error learning , 2014, Behavioural Processes.

[47]  Federico Sanabria,et al.  The distribution of response bout lengths and its sensitivity to differential reinforcement. , 2015, Journal of the experimental analysis of behavior.

[48]  Elliot A. Ludvig,et al.  Timescale invariance in the pacemaker-accumulator family of timing models , 2013 .

[49]  R. Shull,et al.  Bouts of responding: the relation between bout rate and the rate of variable-interval reinforcement. , 2004, Journal of the experimental analysis of behavior.

[50]  Gabriel J. Mazur,et al.  Detrimental effects of acute nicotine on the response-withholding performance of spontaneously hypertensive and Wistar Kyoto rats , 2014, Psychopharmacology.

[51]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[52]  R. Church,et al.  A modular theory of learning and performance , 2007, Psychonomic bulletin & review.

[53]  K. Berridge,et al.  Neuroscience of affect: brain mechanisms of pleasure and displeasure , 2013, Current Opinion in Neurobiology.

[54]  Carter W. Daniels,et al.  Revisiting the effect of nicotine on interval timing , 2015, Behavioural Brain Research.

[55]  W. Meck,et al.  Reinforcement-induced within-trial resetting of an internal clock , 1999, Behavioural Processes.

[56]  M. Stone An Asymptotic Equivalence of Choice of Model by Cross‐Validation and Akaike's Criterion , 1977 .

[57]  I. J. Myung,et al.  Tutorial on maximum likelihood estimation , 2003 .

[58]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[59]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[60]  Marco Vasconcelos,et al.  Animal timing: a synthetic approach , 2016, Animal Cognition.

[61]  R. Shull,et al.  Response rate viewed as engagement bouts: effects of relative reinforcement and schedule type. , 2001, Journal of the experimental analysis of behavior.

[62]  J. Wearden,et al.  “The stone which the builders rejected…”: Delay of reinforcement and response rate on fixed-interval and related schedules , 2006, Behavioural Processes.

[63]  Kimberly S. Kirkpatrick,et al.  Packet theory of conditioning and timing , 2002, Behavioural Processes.

[64]  P. Killeen Reinforcement frequency and contingency as factors in fixed-ratio behavior. , 1969, Journal of the experimental analysis of behavior.

[65]  L. Bizo,et al.  The behavioral theory of timing: Reinforcer rate determines pacemaker rate. , 1994, Journal of the experimental analysis of behavior.

[66]  J. Staddon,et al.  The "supersitition" experiment: A reexamination of its implications for the principles of adaptive behavior. , 1971 .

[67]  Lewis A. Bizo,et al.  Reinforcement context and pacemaker rate in the behavioral theory of timing , 1995 .

[68]  Carter W. Daniels,et al.  Biasing Temporal Judgments in Rats, Pigeons, and Humans , 2015 .

[69]  P. Shizgal,et al.  Operant tempo varies with reinforcement rate: implications for measurement of reward efficacy , 2001, Behavioural Processes.

[70]  R. Shull,et al.  Response rate viewed as engagement bouts: resistance to extinction. , 2002, Journal of the experimental analysis of behavior.

[71]  Federico Sanabria,et al.  An affordance analysis of unconditioned lever pressing in rats and hamsters , 2013, Behavioural Processes.

[72]  J. Horvitz,et al.  Amphetamine affects the start of responding in the peak interval timing task , 2007, Behavioural Processes.

[73]  Jonathan D. Cohen,et al.  Interval Timing by Long-Range Temporal Integration , 2011, Front. Integr. Neurosci..

[74]  P. Killeen Mathematical principles of reinforcement , 1994 .

[75]  Peter R. Killeen,et al.  Temporal generalization accounts for response resurgence in the peak procedure , 2007, Behavioural Processes.

[76]  C. Lowe,et al.  Temporal control of behavior and the power law. , 1979, Journal of the experimental analysis of behavior.

[77]  Andrew T. Marshall,et al.  Motivation and timing: Clues for modeling the reward system , 2012, Behavioural Processes.

[78]  R L Shull Sequential patterns in post-reinforcement pauses on fixed-interval schedules of food. , 1971, Journal of the experimental analysis of behavior.

[79]  R. Rescorla Inhibition of delay in Pavlovian fear conditioning. , 1967, Journal of comparative and physiological psychology.

[80]  R. Church,et al.  Application of scalar timing theory to individual trials. , 1994 .

[81]  J. Gibbon Dynamics of time matching: Arousal makes better seem worse , 1995, Psychonomic bulletin & review.

[82]  P. Killeen,et al.  A behavioral theory of timing. , 1988, Psychological review.

[83]  P. Killeen,et al.  Molecular analyses of the principal components of response strength. , 2002, Journal of the experimental analysis of behavior.

[84]  S. Roberts,et al.  Timing and the control of variation. , 2001, Journal of experimental psychology. Animal behavior processes.

[85]  B. Roche,et al.  The Behavior of Organisms? , 1997 .

[86]  Federico Sanabria,et al.  Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat , 2012, Behavioral and Brain Functions.

[87]  J. J. Higa,et al.  Rapid timing of a single transition in interfood interval duration by rats , 1997 .

[88]  Catalin V Buhusi,et al.  Interval timing as an emergent learning property. , 2003, Psychological review.

[89]  M. Branch,et al.  Effects of cocaine on performance under fixed-interval schedules with a small tandem ratio requirement. , 2004, Journal of the experimental analysis of behavior.

[90]  Federico Sanabria,et al.  Extinction under a behavioral microscope: Isolating the sources of decline in operant response rate , 2012, Behavioural Processes.

[91]  R. Shull,et al.  Bouts of responding from variable-interval reinforcement of lever pressing by rats. , 2003, Journal of the experimental analysis of behavior.

[92]  R. Church,et al.  How rats combine temporal cues , 2005, Behavioural Processes.

[93]  Patrick Simen,et al.  Optimal response rates in humans and rats. , 2015, Journal of experimental psychology. Animal learning and cognition.

[94]  A. Mika,et al.  Chronic stress impairs prefrontal cortex-dependent response inhibition and spatial working memory. , 2012, Behavioral neuroscience.

[95]  S. Hanson,et al.  Measurement and modeling of behavior under fixed-interval schedules of reinforcement. , 1981 .

[96]  Kimberly S. Kirkpatrick,et al.  Reward value effects on timing in the peak procedure , 2009 .

[97]  Josey Y. M. Chu,et al.  The failure of Weber's law in time perception and production , 2006, Behavioural Processes.

[98]  P. Killeen,et al.  Adjunctive behaviors are operants , 2013, Learning & Behavior.

[99]  Peter R. Killeen,et al.  Behavior's Time , 1991 .

[100]  J H Wearden,et al.  Temporal bisection in children. , 2001, Journal of experimental child psychology.

[101]  Kimberly S. Kirkpatrick,et al.  The role of the nucleus accumbens core in impulsive choice, timing, and reward processing. , 2010, Behavioral neuroscience.

[102]  M. Treisman Temporal discrimination and the indifference interval. Implications for a model of the "internal clock". , 1963, Psychological monographs.

[103]  P. Killeen,et al.  Timing with opportunity cost: Concurrent schedules of reinforcement improve peak timing , 2009, Learning & behavior.

[104]  A. Brownstein,et al.  The relative proximity principle and the postreinforcement pause , 1975 .

[105]  Within-session modulation of timed anticipatory responding: When to start responding , 2010, Behavioural Processes.

[106]  R. Church,et al.  Integrating timing and conditioning approaches to study behavior. , 2014, Journal of experimental psychology. Animal learning and cognition.

[107]  Elliot A. Ludvig,et al.  The conditions for temporal tracking under interval schedules of reinforcement. , 2004, Journal of experimental psychology. Animal behavior processes.

[108]  J. Staddon Interval timing: memory, not a clock , 2005, Trends in Cognitive Sciences.

[109]  Elliot A. Ludvig,et al.  The effects of reinforcer magnitude on timing in rats. , 2007, Journal of the experimental analysis of behavior.

[110]  E. Kandel,et al.  Improving Temporal Cognition by Enhancing Motivation Experiment 1: D2r Expression and Timing Method and Materials Subjects Operant Lever-press Training Temporal Discrimination Training , 2022 .

[111]  U. Karmarkar,et al.  Timing in the Absence of Clocks: Encoding Time in Neural Network States , 2007, Neuron.

[112]  Fuat Balci,et al.  Motivational effects on interval timing in dopamine transporter (DAT) knockdown mice , 2010, Brain Research.

[113]  B. Weiss,et al.  The microanalysis of fixed-interval responding. , 1983, Journal of the experimental analysis of behavior.