Deconstructing the law of effect

Abstract Do the consequences of past behavior alter future policy, as the law of effect assumes? Or, are behavioral policies based on behaviorally produced information about the state of the world, but not themselves subject to change? In the first case, stable policies are equilibria discovered by trial and error, so adjustments to abrupt changes in the environment must proceed slowly. In the second, adjustments can be as abrupt as the environmental changes. Matching behavior is the robust tendency of subjects to match the relative time and effort they invest in different foraging options to the relative incomes derived from them. Measurement of the time course of adjustments to step changes in the reward-scheduling environment show that adjustments can be as abrupt as the changes that drive them, and can occur with the minimum possible latency. Broader implications for theories about the role of experience in behavior are discussed.

[1]  G M Heyman,et al.  A Markov model description of changeover probabilities on concurrent variable-interval schedules. , 1979, Journal of the experimental analysis of behavior.

[2]  D. Harper Competitive foraging in mallards: “Ideal free’ ducks , 1982, Animal Behaviour.

[3]  P. Shizgal Neural basis of utility estimation , 1997, Current Opinion in Neurobiology.

[4]  R. Herrnstein,et al.  Maximizing and matching on concurrent ratio schedules. , 1975, Journal of the experimental analysis of behavior.

[5]  R. Luce,et al.  Operant matching is not a logical consequence of maximizing reinforcement rate , 1979 .

[6]  R. Herrnstein,et al.  Melioration: A Theory of Distributed Choice , 1991 .

[7]  C. Gallistel,et al.  The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. , 2001, Journal of experimental psychology. Animal behavior processes.

[8]  M. Davison,et al.  The matching law: A research review. , 1988 .

[9]  C. Gallistel,et al.  Time, rate, and conditioning. , 2000, Psychological review.

[10]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[11]  James F. Wittenberger The Evolution of Mating Systems in Birds and Mammals , 1979 .

[12]  S. Fretwell,et al.  On territorial behavior and other factors influencing habitat distribution in birds , 1969 .

[13]  Richard J. Herrnstein,et al.  MAXIMIZING AND MATCHING ON CONCURRENT RATIO SCHEDULES1 , 1975 .

[14]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[15]  R J HERRNSTEIN,et al.  Relative and absolute strength of response as a function of frequency of reinforcement. , 1961, Journal of the experimental analysis of behavior.

[16]  R. Herrnstein Experiments on Stable Suboptimality in Individual Behavior , 1991 .

[17]  S. Lea,et al.  The Integration of Reinforcements over Time , 1984, Annals of the New York Academy of Sciences.

[18]  J. Godin,et al.  Foraging on patchily distributed prey by a cichlid fish (Teleostei, Cichlidae): A test of the ideal free distribution theory , 1984, Animal Behaviour.

[19]  J E Staddon,et al.  Quasi-dynamic choice models: Melioration and ratio invariance. , 1988, Journal of the experimental analysis of behavior.

[20]  J. Gibbon Dynamics of time matching: Arousal makes better seem worse , 1995, Psychonomic bulletin & review.

[21]  C. Gallistel,et al.  The Symbolic Foundations of Conditioned Behavior , 2002 .

[22]  C. Gallistel Conditioning from an information processing perspective , 2003, Behavioural Processes.