Intermittent Reinforcement and the Persistence of Behavior: Experimental Evidence

Whereas economists have made extensive studies of the impact of levels of incentives on behavior, they have paid little attention to the effects of regularity and frequency of incentives. We contrasted three ways of rewarding participants in a real-effort experiment in which individuals had to decide when to exit the situation : a continuous reinforcement schedule (all periods paid) ; a fixed intermittent reinforcement schedule (one out of three periods paid) ; and a random intermittent reinforcement schedule (one out of three periods paid on a random basis). In all treatments, monetary rewards were withdrawn after the same unknown number of periods. Overall, intermittent reinforcement leads to more persistence and higher total effort, while participants in the continuous condition exit as soon as payment stops or decrease effort dramatically. Randomness increases the dispersion of effort, inducing both early exiting and persistence in behavior ; overall, it reduces agents' payoffs. Our interpretation is that, in the presence of regime shifts, both the frequency and the randomness of the reinforcement schedules influence adjustments that participants make across time to their reference points in earnings expectations. This could explain why agents persist in activities although they lose money, such as excess trading in stock markets.

[1]  M. Villeval,et al.  Lies and Biased Evaluation: A Real-Effort Experiment , 2011, SSRN Electronic Journal.

[2]  Floyd C. Mace,et al.  Schedules of reinforcement , 2011 .

[3]  I. Erev,et al.  Continuous punishment and the potential of gentle rule enforcement , 2010, Behavioural Processes.

[4]  Peter Stone,et al.  Reinforcement learning , 2019, Scholarpedia.

[5]  P. Montague,et al.  Theoretical and Empirical Studies of Learning , 2009 .

[6]  Anders U. Poulsen,et al.  Marriage, partnership and sexual orientation: a study of British university academics and administrators , 2008, Journal of Human Capital.

[7]  Colin Camerer,et al.  Neuroeconomics: decision making and the brain , 2008 .

[8]  L. Goette,et al.  Incentives and the Allocation of Effort Over Time: The Joint Role of Affective and Cognitive Decision Making , 2006, SSRN Electronic Journal.

[9]  D. Hantula,et al.  Equivocality and Escalation: A Replication and Preliminary Examination of Frustration1 , 2005 .

[10]  Cade Massey,et al.  Detecting Regime Shifts: The Causes of Under- and Over-Reaction , 2004, Manag. Sci..

[11]  M. Rabin,et al.  A Model of Reference-Dependent Preferences , 2006 .

[12]  Ben Greiner,et al.  An Online Recruitment System for Economic Experiments , 2004 .

[13]  David Huffman,et al.  Loss Aversion and Labor Supply , 2003, SSRN Electronic Journal.

[14]  Eugene J. Kutcher,et al.  When success breeds failure: history, hysteresis, and delayed exit decisions. , 2003, The Journal of applied psychology.

[15]  E. Fehr,et al.  Do Workers Work More If Wages are High? Evidence from a Randomized Field Experiment , 2005 .

[16]  Colin Camerer,et al.  The Effects of Financial Incentives in Experiments: A Review and Capital-Labor-Production Framework , 1999 .

[17]  Donald A. Hantula,et al.  The Effects of Feedback Equivocality on Escalation of Commitment: An Empirical Investigation of Decision Dilemma Theory1 , 1999 .

[18]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[19]  D. Hantula,et al.  Hyteresis and Uncertainty: The Effect of Uncertainty on Delays to Exit Decisions. , 1998, Organizational behavior and human decision processes.

[20]  Terrance Odean Do Investors Trade Too Much? , 1998 .

[21]  R. Thaler,et al.  Labor Supply of New York City Cabdrivers: One Day at a Time , 1997 .

[22]  S. Kahng,et al.  Responding maintained by intermittent reinforcement: implications for the use of extinction with problem behavior in clinical settings. , 1996, Journal of applied behavior analysis.

[23]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[24]  Craig R. Fox,et al.  Ambiguity Aversion and Comparative Ignorance , 1995 .

[25]  Charles R. Crowell,et al.  Intermittent Reinforcement and Escalation Processes in Sequential Decision Making , 1994 .

[26]  Paul M. B. Vitányi,et al.  Theories of learning , 2007 .

[27]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[28]  B. O'Flaherty,et al.  Going beyond with Bayesian updating. , 1992, Journal of applied behavior analysis.

[29]  S. Goltz,et al.  A sequential learning analysis of decisions in organizations to escalate investments despite continuing costs or losses. , 1992, Journal of applied behavior analysis.

[30]  Avinash Dixit,et al.  Investment and Hysteresis , 1992 .

[31]  J. Brockner The Escalation of Commitment to a Failing Course of Action: Toward Theoretical Progress , 1992 .

[32]  E. Lazear Labor Economics and the Psychology of Organizations , 1991 .

[33]  Henry L. Tosi A Theory of Goal Setting and Task Performance , 1991 .

[34]  E. Lazear The timing of raises and other payments , 1990 .

[35]  Barry M. Staw,et al.  Understanding Behavior in Escalation Situations , 1989, Science.

[36]  Michael G. Bowen The Escalation Phenomenon Reconsidered: Decision Dilemmas or Decision Errors? , 1987 .

[37]  John R. Krebs,et al.  Foraging in a changing environment: An experiment with starlings ("sturnus vulgaris"). , 1987 .

[38]  Peter B. Everett,et al.  EFFECTS OF INTERMITTENT AND CONTINUOUS TOKEN REINFORCEMENT ON BUS RIDERSHIP , 1977 .

[39]  Barry M. Staw,et al.  Knee-deep in the Big Muddy: A study of escalating commitment to a chosen course of action. , 1976 .