Exposing the behavioral gambit: the evolution of learning and decision rules

Behavioral ecologists have long been comfortable assuming that genetic architecture does not constrain which phenotypescan evolve (the "phenotypic gambit"). For flexible behavioral traits, however, solutions to adaptive problems are reached not only by genetic evolution but also by behavioral changes within an individual's lifetime, via psychological mechanisms such as learning. Standard optimality approaches ignore these mechanisms, implicitly assuming that they do not constrain the expression of adaptive behavior. This assumption, which we dub the behavioral gambit, is sometimes wrong: evolved psychological mechanisms can prevent animals from behaving optimally in specific situations. To understand the functional basis of behavior, we would do better by considering the underlying mechanisms, rather than the behavioral outcomes they produce, as the target of selection. This change of focus yields new, testable predictions about evolutionary equilibria, the development of behavior, and the properties of cognitive systems. Studies on the evolution of learning rules hint at the potential insights to be gained, but such mechanism-based approaches are underexploited. We highlight three future research priorities: (1) systematic theoretical analysis of the evolutionary properties of learning rules; (2) detailed empirical study of how animals learn in nonforaging contexts;and (3) analysis of individual differences in learning rules and their associated fitness consequences.

[1]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[2]  R J HERRNSTEIN,et al.  Relative and absolute strength of response as a function of frequency of reinforcement. , 1961, Journal of the experimental analysis of behavior.

[3]  H Rachlin,et al.  Commitment, choice and self-control. , 1972, Journal of the experimental analysis of behavior.

[4]  J M Smith,et al.  Evolution and the theory of games , 1976 .

[5]  R. Lewontin ‘The Selfish Gene’ , 1977, Nature.

[6]  J. Gibbon Scalar expectancy theory and Weber's law in animal timing. , 1977 .

[7]  G M Heyman,et al.  Is matching compatible with reinforcement maximization on concurrent variable interval variable ratio? , 1979, Journal of the experimental analysis of behavior.

[8]  J M Smith Game theory and the evolution of behaviour , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[9]  A Houston,et al.  The application of statistical decision theory to animal behaviour. , 1980, Journal of theoretical biology.

[10]  R. Herrnstein,et al.  CHAPTER 5 – Melioration and Behavioral Allocation1 , 1980 .

[11]  R. Sibly,et al.  Producers and scroungers: A general model and its application to captive flocks of house sparrows , 1981, Animal Behaviour.

[12]  C. Harley Learning the evolutionarily stable strategy. , 1981, Journal of theoretical biology.

[13]  A I Houston,et al.  How to maximize reward rate on two variable-interval paradigms. , 1981, Journal of the experimental analysis of behavior.

[14]  Elizabeth D. Capaldi,et al.  Conditioned flavor preferences based on hunger level during original flavor exposure , 1983 .

[15]  A. Houston Comments on "Learning the evolutionarily stable strategy". , 1983, Journal of theoretical biology.

[16]  S. Lea,et al.  Optimization and flexibility , 1984, Behavioral and Brain Sciences.

[17]  The control of foraging decisions , 1984, Behavioural Processes.

[18]  R. Selten,et al.  Gaps in Harley's argument on evolutionarily stable learning rules and in the logic of “tit for tat” , 1984, Behavioral and Brain Sciences.

[19]  J. L. Gould,et al.  TIT FOR TAT , 1984 .

[20]  A. Grafen Natural selection, kin selection and group selection [Polistes fuscatus, wasps] , 1984 .

[21]  T. Caro,et al.  Organization and ontogeny of alternative tactics , 1986, Animal Behaviour.

[22]  G M Heyman,et al.  OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR MORE ON CONCURRENT INTERVAL-RATIO SCHEDULES : A REPLICATION AND REVIEW GENE , 2005 .

[23]  J. Kagel,et al.  When foragers discount the future: constraint or adaptation? , 1986, Animal Behaviour.

[24]  Alasdair I. Houston,et al.  Learning rules, matching and frequency dependence , 1987 .

[25]  M. Davison,et al.  The matching law: A research review. , 1988 .

[26]  J. Cheverud,et al.  A COMPARISON OF GENETIC AND PHENOTYPIC CORRELATIONS , 1988, Evolution; international journal of organic evolution.

[27]  Alasdair I. Houston,et al.  The value of food: effects of open and closed economies , 1989, Animal Behaviour.

[28]  J. M. Smith,et al.  Optimality theory in evolutionary biology , 1990, Nature.

[29]  A. Houston,et al.  Genetic algorithms and evolution. , 1990, Journal of theoretical biology.

[30]  R. Herrnstein Rational Choice Theory Necessary but Not Sufficient , 1990 .

[31]  M. Nowak,et al.  An evolutionarily stable strategy may be inaccessible. , 1990, Journal of theoretical biology.

[32]  M. Wade,et al.  Equal mating success among male reproductive strategies in a marine isopod , 1991, Nature.

[33]  A. Kacelnik,et al.  Psychological mechanisms and the Marginal Value Theorem: effect of variability in travel time on patch exploitation , 1992, Animal Behaviour.

[34]  A. Kacelnik,et al.  Psychological mechanisms and the Marginal Value Theorem: dynamics of scalar memory for travel time , 1993, Animal Behaviour.

[35]  M. Hori,et al.  Frequency-Dependent Natural Selection in the Handedness of Scale-Eating Cichlid Fish , 1993, Science.

[36]  P. R. Wiepkema,et al.  Consistent individual differences in early exploratory behaviour of male great tits , 1994, Animal Behaviour.

[37]  M. Gross,et al.  The evolution of behavioural ecology. , 1994, Trends in ecology & evolution.

[38]  D. Roff The estimation of genetic correlations from phenotypic correlations: a test of Cheverud's conjecture , 1995, Heredity.

[39]  Joseph Persky,et al.  The Ethology of Homo Economicus , 1995 .

[40]  Laxmaiah Manchikanti,et al.  An Invited Review , 1995 .

[41]  D. Stephens,et al.  Testing models of non-kin cooperation: mutualism and the Prisoner’s Dilemma , 1995, Animal Behaviour.

[42]  J W Seaman,et al.  Properties of evolutionarily stable learning rules. , 1995, Journal of theoretical biology.

[43]  J. Gibbon,et al.  Memory for inter-reinforcement interval variability and patch departure decisions in the starling, Sturnus vulgaris , 1996, Animal Behaviour.

[44]  F. Weissing,et al.  Genetic versus phenotypic models of selection: can genetics be neglected in a long-term perspective? , 1996, Journal of mathematical biology.

[45]  B. Sinervo,et al.  The rock–paper–scissors game and the evolution of alternative male strategies , 1996, Nature.

[46]  A. Rubinstein Modeling Bounded Rationality , 1998 .

[47]  M. Stamp Dawkins,et al.  Cognitive ecology: the evolutionary ecology of information processing and decision making. , 1998, Trends in cognitive sciences.

[48]  R. Dukas,et al.  Cognitive ecology : the evolutionary ecology of information processing and decision making , 1998 .

[49]  S. Shettleworth Cognition, evolution, and behavior , 1998 .

[50]  Alasdair I Houston,et al.  Incorporating rules for responding into evolutionary games , 1999, Nature.

[51]  Seth Bullock,et al.  Simple Heuristics That Make Us Smart , 1999 .

[52]  P. Todd,et al.  From pride and prejudice to persuasion : Satisficing in mate search , 1999 .

[53]  D. Réale,et al.  Quantitative genetics of life-history traits in a long-lived wild mammal , 2000, Heredity.

[54]  G. Beauchamp,et al.  Learning rules for social foragers: implications for the producer-scrounger game and ideal free distribution theory. , 2000, Journal of theoretical biology.

[55]  P. Bednekoff Models of Adaptive Behaviour: An Approach based on State , 2000, Animal Behaviour.

[56]  T. Caraco,et al.  Social Foraging Theory , 2018 .

[57]  J. E. Mazur Tradeoffs among delay, rate, and amount of reinforcement , 2000, Behavioural Processes.

[58]  L. Giraldeau,et al.  Experimental evidence that group foragers can converge on predicted producer–scrounger equilibria , 2000, Animal Behaviour.

[59]  G. Loewenstein Emotions in Economic Theory and Economic Behavior , 2000 .

[60]  Colin Camerer,et al.  In search of homo economicus: Experiments in 15 small-scale societies , 2001 .

[61]  R. Boyd,et al.  In Search of Homo Economicus: Behavioral Experiments in 15 Small- Scale Societies , 2001 .

[62]  R. Gray,et al.  Cycles of Contingency: Developmental Systems and Evolution , 2001 .

[63]  Patrick Bateson,et al.  Behavioral development and Darwinian evolution. , 2001 .

[64]  S. Shafir,et al.  Context-dependent violations of rational choice in honeybees (Apis mellifera) and gray jays (Perisoreus canadensis) , 2001, Behavioral Ecology and Sociobiology.

[65]  Melissa Bateson,et al.  Context-dependent foraging choices in risk-sensitive starlings , 2002, Animal Behaviour.

[66]  D. Stephens,et al.  Discrimination, discounting and impulsivity: a role for an informational constraint. , 2002, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[67]  Stephen F. Davis,et al.  Handbook of Research Methods in Experimental Psychology , 2003 .

[68]  J. Stamps Behavioural processes affecting development: Tinbergen's fourth question comes of age , 2003, Animal Behaviour.

[69]  D. Kahneman Maps of Bounded Rationality: Psychology for Behavioral Economics , 2003 .

[70]  O. Curry Bounded Rationality : The Adaptive Toolbox , 2003 .

[71]  Martin Guha Handbook of Research Methods in Experimental Psychology , 2003 .

[72]  Estimating mechanisms and equilibria for offspring begging and parental provisioning , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[73]  T. A. Hurly,et al.  Context–dependent foraging decisions in rufous hummingbirds , 2003, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[74]  Troy Day,et al.  Stability in negotiation games and the emergence of cooperation† , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[75]  M. Mendl,et al.  Animal behaviour: Cognitive bias and affective state , 2004, Nature.

[76]  A. Kacelnik,et al.  State-Dependent Decisions Cause Apparent Violations of Rationality in Animal Choice , 2004, PLoS biology.

[77]  D. Stephens,et al.  The economic basis of cooperation: tradeoffs between selfishness and generosity , 2004 .

[78]  D. Stephens,et al.  Impulsiveness without discounting: the ecological rationality hypothesis , 2004, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[79]  Patrick Bateson,et al.  The Active Role of Behaviour in Evolution , 2004 .

[80]  Alex Kacelnik,et al.  Energetic state during learning affects foraging choices in starlings , 2004 .

[81]  Hanna Damasio,et al.  Investment Behavior and the Negative Side of Emotion , 2005, Psychological science.

[82]  D. DeAngelis,et al.  Individual-based modeling of ecological and evolutionary processes , 2005 .

[83]  J. Hutchinson,et al.  Simple heuristics and rules of thumb: Where psychologists and behavioural biologists might meet , 2005, Behavioural Processes.

[84]  Melissa Bateson,et al.  Comparative evaluation and its implications for mate choice. , 2005, Trends in ecology & evolution.

[85]  A. Kacelnik Meanings of rationality , 2006 .

[86]  L. L. Wolf,et al.  Modulation of aggressive behaviour by fighting experience: mechanisms and contest outcomes , 2005, Biological reviews of the Cambridge Philosophical Society.

[87]  Alex Kacelnik,et al.  State-Dependent Learned Valuation Drives Choice in an Invertebrate , 2006, Science.

[88]  R. Johnstone,et al.  Negotiation over Offspring Care—how Should Parents Respond to Each Other's Efforts? , 2006 .

[89]  J. Hadfield,et al.  Testing the phenotypic gambit: phenotypic, genetic and environmental correlations of colour , 2007, Journal of evolutionary biology.

[90]  David W. Stephens,et al.  The discounting-by-interruptions hypothesis: model and experiment , 2007 .

[91]  N. Dingemanse,et al.  Integrating animal temperament within ecology and evolution , 2007, Biological reviews of the Cambridge Philosophical Society.

[92]  Steven Hamblin,et al.  Genetic algorithms and non-ESS solutions to game theory models , 2007, Animal Behaviour.

[93]  Alasdair I Houston,et al.  Do we expect natural selection to produce rational behaviour? , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[94]  Graeme D Ruxton,et al.  The application of genetic algorithms in behavioural ecology, illustrated with a model of anti-predator vigilance. , 2008, Journal of theoretical biology.

[95]  Melissa Bateson,et al.  Larger, enriched cages are associated with 'optimistic' response biases in captive European starlings (Sturnus vulgaris) , 2008 .

[96]  Roderich Groß,et al.  Simple learning rules to cope with changing environments , 2008, Journal of The Royal Society Interface.

[97]  Yael Niv,et al.  Operant Conditioning , 1971 .

[98]  Animal Learning and Animal Cognition , 2008 .

[99]  L. Giraldeau,et al.  Chapter 2 Social Foraging and the Study of Exploitative Behavior , 2008 .

[100]  Jeffrey R. Stevens,et al.  The evolutionary biology of decision making , 2008 .

[101]  Franz J. Weissing,et al.  Evolutionary emergence of responsive and unresponsive personalities , 2008, Proceedings of the National Academy of Sciences.

[102]  Jonathan Wright,et al.  Parent–offspring conflict and co-adaptation: behavioural ecology meets quantitative genetics , 2008, Proceedings of the Royal Society B: Biological Sciences.

[103]  W. Singer,et al.  Better than conscious? : decision making, the human mind, and implications for institutions , 2008 .

[104]  A. Kacelnik,et al.  Simultaneous and sequential choice as a function of reward delay and magnitude: normative, descriptive and process-based models tested in the European starling (Sturnus vulgaris). , 2008, Journal of experimental psychology. Animal behavior processes.

[105]  Ellison,et al.  Endocrinology of social relationships , 2008 .

[106]  Alasdair I. Houston,et al.  Flying in the face of nature , 2009, Behavioural Processes.

[107]  Alex Kacelnik,et al.  State-dependent valuation learning in fish: Banded tetras prefer stimuli associated with greater past deprivation , 2009, Behavioural Processes.

[108]  A. Houston,et al.  Integrating function and mechanism. , 2009, Trends in ecology & evolution.

[109]  Luc-Alain Giraldeau,et al.  Finding the evolutionarily stable learning rule for frequency-dependent foraging , 2009, Animal Behaviour.

[110]  Andreas Wilke,et al.  Adaptive rationality: An evolutionary perspective on cognitive bias , 2009 .

[111]  R. Dukas,et al.  Cognitive ecology II , 2009 .

[112]  Andreas Wilke,et al.  The hot hand phenomenon as a cognitive adaptation to clumped resources , 2009 .

[113]  M. Kölliker,et al.  Behavioural attainability of evolutionarily stable strategies in repeated interactions , 2009, Animal Behaviour.

[114]  K. Larose,et al.  Long-term social bonds promote cooperation in the iterated Prisoner's Dilemma , 2009, Proceedings of the Royal Society B: Biological Sciences.

[115]  L. Giraldeau,et al.  Learning behaviorally stable solutions to producer–scrounger games , 2010 .

[116]  Bart Kempenaers,et al.  Spatial autocorrelation: an overlooked concept in behavioral ecology , 2010, Behavioral ecology : official journal of the International Society for Behavioral Ecology.

[117]  T. W. Fawcett,et al.  change with age and experience Learning your own strength : winner and loser effects should , 2010 .

[118]  D. Puts The psychologic gambit declined—a review of “Endocrinology of Social Relationships” , 2010 .

[119]  N. Tinbergen On aims and methods of Ethology , 2010 .

[120]  Peter DeScioli Heavy hearts and heads held high — A review of Glimpses of Creatures in Their Physical Worlds , 2010 .

[121]  S. Shettleworth Cognition, evolution, and behavior, 2nd ed. , 2010 .

[122]  Eva Maria Buchkremer,et al.  The emergence of variance-sensitivity with successful decision rules , 2010 .

[123]  Theodore P. Pavlic,et al.  When rate maximization is impulsive , 2010, Behavioral Ecology and Sociobiology.

[124]  Richard E Tremblay,et al.  Developmental origins of disruptive behaviour problems: the 'original sin' hypothesis, epigenetics and their consequences for prevention. , 2010, Journal of child psychology and psychiatry, and allied disciplines.

[125]  Denis Réale,et al.  Behavioural reaction norms: animal personality meets individual plasticity. , 2010, Trends in ecology & evolution.

[126]  I. Wang,et al.  REVIEW: Optimality models in the age of experimental evolution and genomics , 2010, Journal of evolutionary biology.

[127]  Julie Morand-Ferron,et al.  Learning in a game context: strategy choice by some keeps learning from evolving in others , 2010, Proceedings of the Royal Society B: Biological Sciences.

[128]  M. Bateson,et al.  Agitated Honeybees Exhibit Pessimistic Cognitive Biases , 2011, Current Biology.

[129]  K. Laland,et al.  Cause and Effect in Biology Revisited: Is Mayr's Proximate-ultimate , 2011 .