Learning rules, matching and frequency dependence

We believe that Harley (1981, 1983) confuses various issues that are important in the application of optimality and game theory principles to behaviour. This paper attempts to clarify these issues, using examples based on various studies of foraging behaviour. We argue that it is important to distinguish between two meanings of “frequency dependent”. One meaning is concerned with how the payoff to a given action in an animal's repertoire depends on the frequency with which the animal uses the action. The other meaning is concerned with how the payoff to a strategy depends on the frequency with which the strategy is present in the population. A confusion between these levels seems to underlie an incorrect proof that the matching law is optimal.

[1]  A. Houston.  Comments on "Learning the evolutionarily stable strategy". , 1983, Journal of theoretical biology.

[2]  Manfred Milinski,et al.  Competitive resource sharing: An experimental test of a learning rule for ESSs , 1984, Animal Behaviour.

[3]  R J HERRNSTEIN,et al.  Relative and absolute strength of response as a function of frequency of reinforcement. , 1961, Journal of the experimental analysis of behavior.

[4]  Klaus Regelmann,et al.  Competitive resource sharing: A simulation model , 1984, Animal Behaviour.

[5]  C B Harley,et al.  When do animals learn the evolutionarily stable strategy? , 1983, Journal of theoretical biology.

[6]  A I Houston,et al.  How to maximize reward rate on two variable-interval paradigms. , 1981, Journal of the experimental analysis of behavior.

[7]  John Maynard Smith,et al.  Game theory and the evolution of behaviour , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[8]  E. Charnov.  Optimal foraging, the marginal value theorem. , 1976, Theoretical population biology.

[9]  C. Harley.  Learning the evolutionarily stable strategy. , 1981, Journal of theoretical biology.

[10]  R. Luce,et al.  Operant matching is not a logical consequence of maximizing reinforcement rate , 1979 .

[11]  R. Selten,et al.  Gaps in Harley's argument on evolutionarily stable learning rules and in the logic of “tit for tat” , 1984, Behavioral and Brain Sciences.

[12]  W. Baum,et al.  Matching, undermatching, and overmatching in studies of choice. , 1979, Journal of the experimental analysis of behavior.

[13]  J D Findley,et al.  Preference and Switching under Concurrent Scheduling. , 1958, Journal of the experimental analysis of behavior.

[14]  Drazen Prelec,et al.  Matching, maximizing, and the hyperbolic reinforcement feedback function. , 1982 .

[15]  J. M. Smith.  The theory of games and the evolution of animal conflicts. , 1974, Journal of theoretical biology.

[16]  J. Staddon,et al.  On matching and maximizing in operant choice experiments. , 1978 .

[17]  S. Fretwell,et al.  On territorial behavior and other factors influencing habitat distribution in birds , 1969 .

[18]  H Rachlin,et al.  Matching and maximizing with concurrent ratio-interval schedules. , 1983, Journal of the experimental analysis of behavior.

[19]  R. Herrnstein.  On the law of effect. , 1970, Journal of the experimental analysis of behavior.

[20]  G. Pyke.  Optimal foraging in hummingbirds : testing the marginal value theorem , 1978 .

[21]  D. Stubbs,et al.  Concurrent responding with fixed relative rate of reinforcement. , 1969, Journal of the experimental analysis of behavior.

[22]  J. E. Mazur.  Is matching behavior an evolutionary inevitability? , 1984, Behavioral and Brain Sciences.

[23]  W M Baum,et al.  On two types of deviation from the matching law: bias and undermatching. , 1974, Journal of the experimental analysis of behavior.

[24]  A. Houston,et al.  Optimal foraging and learning , 1985 .

[25]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[26]  Geoffrey Parker,et al.  Animal Behavior as a Strategy Optimizer: Evolution of Resource Assessment Strategies and Optimal Emigration Thresholds , 1976, The American Naturalist.

[27]  G M Heyman,et al.  A Markov model description of changeover probabilities on concurrent variable-interval schedules. , 1979, Journal of the experimental analysis of behavior.

[28]  B. Williams.  Choice behavior in a discrete-trial concurrent VI-VR: A test of maximizing theories of matching , 1985 .

[29]  J. M. Smith,et al.  Optimization Theory in Evolution , 1978 .

[30]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[31]  G M Heyman,et al.  Is matching compatible with reinforcement maximization on concurrent variable interval variable ratio? , 1979, Journal of the experimental analysis of behavior.