Reward and punishment act as distinct factors in guiding behavior

Behavior rests on the experience of reinforcement and punishment. It has been unclear whether reinforcement and punishment act as oppositely valenced components of a single behavioral factor, or whether these two kinds of outcomes play fundamentally distinct behavioral roles. To this end, we varied the magnitude of a reward or a penalty experienced following a choice using monetary tokens. The outcome of each trial was independent of the outcome of the previous trial, which enabled us to isolate and study the effect on behavior of each outcome magnitude in single trials. We found that a reward led to a repetition of the previous choice, whereas a penalty led to an avoidance of the previous choice. Surprisingly, the effects of the reward magnitude and the penalty magnitude revealed a pronounced asymmetry. The choice repetition effect of a reward scaled with the magnitude of the reward. In a marked contrast, the avoidance effect of a penalty was flat, not influenced by the magnitude of the penalty. These effects were mechanistically described using a reinforcement learning model after the model was updated to account for the penalty-based asymmetry. The asymmetry in the effects of the reward magnitude and the punishment magnitude was so striking that it is difficult to conceive that one factor is just a weighted or transformed form of the other factor. Instead, the data suggest that rewards and penalties are fundamentally distinct factors in governing behavior.

[1]  Eduardo F. Morales,et al.  An Introduction to Reinforcement Learning , 2011 .

[2]  D. Lerman,et al.  On the status of knowledge for using punishment implications for treating behavior disorders. , 2002, Journal of applied behavior analysis.

[3]  R. Herrnstein On the law of effect. , 1970, Journal of the experimental analysis of behavior.

[4]  A. Sanfey,et al.  Independent Coding of Reward Magnitude and Valence in the Human Brain , 2004, The Journal of Neuroscience.

[5]  P. Hineline Aversive control: A separate domain? , 1984, Journal of the experimental analysis of behavior.

[6]  J. Spence Verbal-discrimination performance as a function of instructions and verbal-reinforcement combination in normal and retarded children. , 1966, Child development.

[7]  Kenneth L. Hoving,et al.  The effectiveness of reward and punishment contingencies on response inhibition , 1973 .

[8]  R. Penney,et al.  Children's discrimination learning as a function of reward and punishment. , 1961, Journal of comparative and physiological psychology.

[9]  W. Meyer,et al.  Effectiveness of reward and punishment as a function of task complexity. , 1962 .

[10]  B. Skinner Operant Behavior , 2021, Encyclopedia of Evolutionary Psychological Science.

[11]  M. Davison,et al.  Choice in a variable environment: every reinforcer counts. , 2000, Journal of the experimental analysis of behavior.

[12]  H. Simon,et al.  Theories of Decision-Making in Economics and Behavioural Science , 1966 .

[13]  J A DINSMOOR,et al.  Punishment. I. The avoidance hypothesis. , 1954, Psychological review.

[14]  Adrian R. Willoughby,et al.  The Medial Frontal Cortex and the Rapid Processing of Monetary Gains and Losses , 2002, Science.

[15]  H. Seo,et al.  Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game , 2009, Journal of Neuroscience.

[16]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[17]  R. Epstein,et al.  The positive side effects of reinforcement: a commentary on Balsam and Bondy (1983). , 1985, Journal of applied behavior analysis.

[18]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[19]  M. C. Newland,et al.  Asymmetry of reinforcement and punishment in human choice. , 2008, Journal of the experimental analysis of behavior.

[20]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[21]  P. Glimcher,et al.  Activity in Posterior Parietal Cortex Is Correlated with the Relative Subjective Desirability of Action , 2004, Neuron.

[22]  Darrell A. Worthy,et al.  Heterogeneity of strategy use in the Iowa gambling task: A comparison of win-stay/lose-shift and reinforcement learning models , 2013, Psychonomic bulletin & review.

[23]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[24]  H. Rachlin Judgment, Decision, and Choice: A Cognitive/Behavioral Synthesis , 1989 .

[25]  A. Raftery Bayesian Model Selection in Social Research , 1995 .

[26]  M. C. Stafford,et al.  Rewards and Punishments in Complex Human Choices , 1991 .

[27]  K. Vohs,et al.  Case Western Reserve University , 1990 .

[28]  Maria Adler,et al.  Science and human behavior , 2017 .

[29]  S. Ian Robertson,et al.  Problem-solving , 2001, Human Thinking.

[30]  Shelley E. Taylor,et al.  Asymmetrical effects of positive and negative events: the mobilization-minimization hypothesis. , 1991, Psychological bulletin.

[31]  Ahmad Shamloo,et al.  Punishment , 1997 .

[32]  E Fantino,et al.  The symmetrical law of effect and the matching relation in choice behavior. , 1978, Journal of the experimental analysis of behavior.

[33]  O. Mowrer On the dual nature of learning—a re-interpretation of "conditioning" and "problem-solving." , 1947 .

[34]  Jerome R. Busemeyer,et al.  Comparison of Decision Learning Models Using the Generalization Criterion Method , 2008, Cogn. Sci..

[35]  W. Brown Animal Intelligence: Experimental Studies , 1912, Nature.

[36]  Michael J. Frank,et al.  Error-Related Negativity Predicts Reinforcement Learning and Conflict Biases , 2005, Neuron.

[37]  C. Bradshaw,et al.  The effect of punishment on free-operant choice behavior in humans. , 1979, Journal of the experimental analysis of behavior.

[38]  R J Herrnstein,et al.  Negative reinforcement as shock-frequency reduction. , 1966, Journal of the experimental analysis of behavior.

[39]  E. Thorndike “Animal Intelligence” , 1898, Nature.

[40]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[41]  J. Gibbon,et al.  Cognition and behavior in studies of choice , 1986 .

[42]  C. Fiorillo Two Dimensions of Value: Dopamine Neurons Represent Reward But Not Aversiveness , 2013, Science.

[43]  Bingni W. Brunton,et al.  A low-frequency oscillatory neural signal in humans encodes a developing decision variable , 2013, NeuroImage.

[44]  I. Ehrlich Crime, Punishment, and the Market for Offenses , 1996 .

[45]  Murray Sidman Reduction of shock frequency as reinforcement for avoidance behavior. , 1962, Journal of the experimental analysis of behavior.

[46]  E. Yechiam,et al.  Losses as modulators of attention: review and analysis of the unique effects of losses over gains. , 2013, Psychological bulletin.

[47]  W. Schultz Behavioral dopamine signals , 2007, Trends in Neurosciences.

[48]  J. Dinsmoor Still No Evidence For Temporally Extended Shock-frequency Reduction As A Reinforcer. , 2001 .

[49]  T. Hackenberg Token reinforcement: a review and analysis. , 2009, Journal of the experimental analysis of behavior.

[50]  Brent Alsop,et al.  Reinforcement and punishment inbehavioral models of signal detection , 2007 .

[51]  R Schuster,et al.  Indifference between punishment and free shock: evidence for the negative law of effect. , 1968, Journal of the experimental analysis of behavior.

[52]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[53]  A. Wierzbicki A Mathematical Basis for Satisficing Decision Making , 1982 .

[54]  M. Davison Choice, changeover, and travel: A quantitative model. , 1991, Journal of the experimental analysis of behavior.

[55]  Edward B. Royzman,et al.  Negativity Bias, Negativity Dominance, and Contagion , 2001 .

[56]  R. G. Ratliff,et al.  Interaction of reinforcement conditions and developmental level in a two-choice discrimination task with children , 1974 .

[57]  D. W. Hands The Matching Law: Papers In Psychology And Economics , 1999 .

[58]  R. Penney,et al.  Effect of reward and punishment on children's orientation and discrimination learning. , 1967, Journal of experimental psychology.

[59]  M. C. Newland,et al.  JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2003, 80, 1–27 NUMBER 1(JULY) PUNISHMENT IN HUMAN CHOICE: DIRECT OR COMPETITIVE SUPPRESSION? , 2022 .

[60]  Y. Niv Reinforcement learning in the brain , 2009 .

[61]  A. Tversky,et al.  Rational choice and the framing of decisions , 1990 .