Matrix methods for stochastic dynamic programming in ecology and evolutionary biology

Funding information This work was supported by the Natural Sciences and Engineering Research Council of Canada, Alberta Innovates, and the Killam Trust through scholarships to J.R.R. M.M. acknowledges NSF grant DEB 1555729 and ONR Grant N00014‐19‐1‐2494. A.E.D. acknowledges support from ArcticNet, Environment and Climate Change Canada, Hauser Bears, Natural Sciences and Engineering Research Council of Canada, Polar Bears International, Polar Continental Shelf Project, Quark Expeditions, and World Wildlife Fund (Canada). M.A.L. gratefully acknowledges an NSERC Discovery Grant and a Canada Research Chair.

[1]  Jan‐Åke Nilsson,et al.  Adaptive temperature regulation in the little bird in winter: predictions from a stochastic dynamic programming model , 2017, Oecologia.

[2]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[3]  Andrew G. Barto,et al.  Enriching behavioral ecology with reinforcement learning methods , 2018, Behavioural Processes.

[4]  Theodore J. Sheskin,et al.  Markov Chains and Decision Processes for Engineers and Managers , 2010 .

[5]  Marie-Josée Cros,et al.  MDPtoolbox: a multi-platform toolbox to solve stochastic dynamic programming problems , 2014 .

[6]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[7]  Michael C. Runge,et al.  THE IMPORTANCE OF FUNCTIONAL FORM IN OPTIMAL CONTROL SOLUTIONS OF PROBLEMS IN POPULATION DYNAMICS , 2002 .

[8]  J. McNamara.  THE POLICY WHICH MAXIMISES LONG-TERM SURVIVAL OF AN ANIMAL FACED WITH THE RISKS OF STARVATION AND PREDATION , 1990 .

[9]  I. Chades,et al.  Beyond stochastic dynamic programming: a heuristic sampling method for optimizing conservation decisions in very large state spaces , 2011 .

[10]  Marc Mangel,et al.  Seasonal matching of foraging to anticipated energy requirements in anorexic juvenile salmon , 1996, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[11]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[12]  S. Wright,et al.  An Analysis of Local Variability of Flower Color in Linanthus Parryae. , 1943, Genetics.

[13]  H. Godfray,et al.  Host-feeding strategies of parasitoid wasps , 1993, Evolutionary Ecology.

[14]  C. Clark,et al.  Dynamic Modeling in Behavioral Ecology , 2019 .

[15]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[16]  Katriona Shea,et al.  A state-dependent model for the optimal management of an invasive metapopulation. , 2008, Ecological applications : a publication of the Ecological Society of America.

[17]  Groupe Pdmia.  Markov Decision Processes In Artificial Intelligence , 2009 .

[18]  Lucile Marescot,et al.  Complex decisions made simple: a primer on stochastic dynamic programming , 2013 .

[19]  M. Mangel.  Evolution of Host Selection in Parasitoids: Does the State of the Parasitoid Matter? , 1989, The American Naturalist.

[20]  Marc Mangel,et al.  Phenotypic Evolutionary Models in Stem Cell Biology: Replacement, Quiescence, and Variability , 2008, PloS one.

[21]  M. Mangel.  Opposition site selection and clutch size in insects , 1987 .

[22]  François Charpillet,et al.  Dynamic optimization over infinite-time horizon: web-building strategy in an orb-weaving spider as a case study. , 2006, Journal of theoretical biology.

[23]  Craig Boutilier,et al.  Stochastic dynamic programming with factored representations , 2000, Artif. Intell..

[24]  Hugh P. Possingham,et al.  Optimal release strategies for biological control agents: an application of stochastic dynamic programming to population management , 2000 .

[25]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[26]  John M. McNamara,et al.  Optimal life histories: a generalisation of the Perran-FrobeniusTheorem , 1991 .

[27]  H. Caswell.  Beyond R 0: Demographic Models for Variability of Lifetime Reproductive Output , 2011, PloS one.

[28]  M. Mangel,et al.  Evolution of Size‐Dependent Flowering in Onopordum illyricum: A Quantitative Assessment of the Role of Stochastic Selection Pressures , 1999, The American Naturalist.

[29]  M. Mangel.  Adaptive walks on behavioural landscapes and the evolution of optimal behaviour by natural selection , 2005, Evolutionary Ecology.

[30]  M. Mangel,et al.  Modeling optimal responses and fitness consequences in a changing Arctic , 2019, Global change biology.

[31]  Edmund J. Collins,et al.  Optimality Models in Behavioral Biology , 2001, SIAM Rev..

[32]  E. Charnov,et al.  Evolution of Host Selection and Clutch Size in Parasitoid Wasps , 1984 .

[33]  H. Caswell,et al.  Lifetime reproductive output: individual stochasticity, variance, and sensitivity analysis , 2017, Theoretical Ecology.

[34]  Marc Mangel,et al.  Stochastic Dynamic Programming Illuminates the Link Between Environment, Physiology, and Evolution , 2015, Bulletin of mathematical biology.

[35]  C. Clark,et al.  Towards a Unifield Foraging Theory , 1986 .

[36]  E. Charnov,et al.  Life-History Consequences of Natural Selection: Cole's Result Revisited , 1973, The American Naturalist.

[37]  A. Houston,et al.  The Common Currency for Behavioral Decisions , 1986, The American Naturalist.

[38]  C. Clark,et al.  Dynamic State Variable Models in Ecology , 2000 .

[39]  J. M. Smith,et al.  Optimality theory in evolutionary biology , 1990, Nature.

[40]  U. Rieder,et al.  Markov Decision Processes with Applications to Finance , 2011 .

[41]  A. Grant,et al.  Life History Evolution , 2002, Heredity.

[42]  T. Dobzhansky,et al.  Genetics of Natural Populations. VI. Microgeographic Races in Linanthus Parryae. , 1942, Genetics.