Dynamic optimization over infinite-time horizon: web-building strategy in an orb-weaving spider as a case study.

Dynamic state-dependent models have been widely developed since 1990s for solving questions in evolutionary ecology. Up to now, these models were mainly run over finite-time horizon. However, for many biological questions an infinite-time horizon perspective could be more appropriate, especially when the end of the modeled period is state- rather than time-dependent. Despite this approach is widely used in the field of economics and operational research, thus far no work has been providing biologists with a general method to solve infinite-time horizon problems. Here we present such a method, through the exhaustive description of an algorithm that we implement to determine the strategy an organism should follow to reach a particular state as fast as possible while limiting mortality risk. To illustrate that method we explored web-building behavior in an orb-weaving spider. How are adult females predicted to build their successive webs to gain energy, grow, and lay their first clutch as fast as possible, without suffering from either predation or starvation? From this example, we first show how an optimal strategy over infinite-time horizon can be processed and selected. Second, we analyse variations of the optimal web-building strategy along with the spider's body weight and predation risk during web building. Our model yields two main predictions: (1) spiders reduce their web size as they are gaining weight due to body-mass-dependent cost of web-building behavior, and (2) this reduction in web size starts at lower weight under higher predation risk.

[1]  Anders Brodin,et al.  Why do hoarding birds gain fat in winter in the wrong way? Suggestions from a dynamic model , 2000 .

[2]  P. Sherman The orb-web: an energetic and behavioural estimator of a spider's dynamic foraging and reproductive strategies , 1994, Animal Behaviour.

[3]  Samuel Venner,et al.  Web-building behaviour in the orb-weaving spider Zygiella x-notata: influence of experience , 2000, Animal Behaviour.

[4]  Linden E. Higgins,et al.  Variation in foraging investment during the intermolt interval and before egg-laying in the spiderNephila clavipes (Araneae: Araneidae) , 1990, Journal of Insect Behavior.

[5]  Marc Mangel,et al.  Dynamic models in behavioural and evolutionary ecology , 1988, Nature.

[6]  A. Houston,et al.  The Common Currency for Behavioral Decisions , 1986, The American Naturalist.

[7]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[8]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[9]  W. Shear,et al.  Spiders : webs, behavior, and evolution , 1986 .

[10]  L. Real Search Theory and Mate Choice. I. Models of Single-Sex Discrimination , 1990, The American Naturalist.

[11]  Samuel Venner,et al.  Body-mass-dependent cost of web-building behavior in an orb weaving spider, Zygiella x-notata , 2003, Naturwissenschaften.

[12]  Alain Pasquet,et al.  Previous foraging success influences web building in the spider Stegodyphus lineatus (Eresidae) , 1999 .

[13]  A. Shwartz,et al.  Handbook of Markov decision processes : methods and applications , 2002 .

[14]  A. Houston,et al.  State-dependent life histories , 1996, Nature.

[15]  R. Bellman Dynamic programming. , 1957, Science.

[16]  Alasdair I Houston,et al.  When to parasitize? A dynamic optimization model of reproductive strategies in a cooperative breeder. , 2004, Journal of theoretical biology.

[17]  M. Cichoń,et al.  Ageing and typical survivorship curves result from optimal resource allocation , 2000 .

[18]  C. Clark,et al.  Towards a Unifield Foraging Theory , 1986 .

[19]  David B. Peakall,et al.  Silk Making. (Book Reviews: A Spider's Web. Problems in Regulatory Biology) , 1968 .

[20]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[21]  Jeffrey R. Lucas,et al.  Daily patterns of energy storage in food-caching birds under variable daily predation risk: a dynamic state variable model , 2001, Behavioral Ecology and Sociobiology.

[22]  Samuel Venner,et al.  Spider webs designed for rare but life-saving catches , 2005, Proceedings of the Royal Society B: Biological Sciences.

[23]  Samuel Venner Stratégies comportementales et modèle d'optimisation dynamique à horizon non fini : succession des constructions de toiles chez une araignée orbitèle "Zygiella X-Notata" (Clerck) , 2002 .

[24]  M. Cichoń,et al.  Diversity of age‐specific reproductive rates may result from ageing and optimal resource allocation , 2001, Journal of evolutionary biology.

[25]  A. Houston,et al.  A model of risk-sensitive foraging for a reproducing animal , 1991, Animal Behaviour.

[26]  S. Venner,et al.  Estimation of the Web’s Capture Thread Length in Orb-Weaving Spiders: Determining the Most Efficient Formula , 2001 .

[27]  P N Witt,et al.  The energy budget of an orb web-building spider. , 1976, Comparative biochemistry and physiology. A, Comparative physiology.

[28]  Koichi Tanaka,et al.  Energetic cost of web construction and its effect on web relocation in the web-building spider Agelena limbata , 1989, Oecologia.

[29]  Alasdair I. Houston,et al.  Optimal avian migration: A dynamic model of fuel stores and site use , 1998, Evolutionary Ecology.

[30]  Marc Mangel,et al.  Effects of Time Limitation and Egg Limitation on Lifetime Reproductive Success of a Parasitoid in the Field , 1998, The American Naturalist.

[31]  Edmund J. Collins,et al.  Optimality Models in Behavioral Biology , 2001, SIAM Rev..

[32]  John M. McNamara,et al.  Optimal life histories: a generalisation of the Perran-FrobeniusTheorem , 1991 .

[33]  K. Prestwich,et al.  The energetics of web-building in spiders , 1977 .

[34]  Alasdair I Houston,et al.  The effects of background mortality on optimal reproduction in a seasonal environment. , 2004, Theoretical population biology.

[35]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[36]  A. Pasquet,et al.  Presence of potential prey affects web-building in an orb-weaving spider Zygiella x-notata , 1994, Animal Behaviour.

[37]  R. Foelix,et al.  The biology of spiders. , 1987 .

[38]  P. Kindlmann,et al.  Dynamics of Production of Sexual Forms in Aphids: Theoretical and Experimental Evidence for Adaptive “Coin‐Flipping” Plasticity , 2004, The American Naturalist.

[39]  M. Herberstein,et al.  The role of experience in web-building spiders (Araneidae) , 1999, Animal Cognition.

[40]  Leslie A. Real,et al.  Search Theory and Mate Choice. II. Mutual Interaction, Assortative Mating, and Equilibrium Variation in Male and Female Fitness , 1991, The American Naturalist.

[41]  C. Clark,et al.  Dynamic State Variable Models in Ecology: Methods and Applications , 2001 .

[42]  L Rogers,et al.  A Dynamic Model of Size‐Dependent Reproductive Effort in a Sequential Hermaphrodite: A Counterexample to Williams's Conjecture , 2001, The American Naturalist.