Selecting simultaneous actions of different durations to optimally manage an ecological network

Species management requires decision‐making under uncertainty. Given a management objective and limited budget, managers need to decide what to do, and where and when to do it. A schedule of management actions that achieves the best performance is an optimal policy. A popular optimisation technique used to find optimal policies in ecology and conservation is stochastic dynamic programming (SDP). Most SDP approaches can only accommodate actions of equal durations. However, in many situations, actions take time to implement or cannot change rapidly. Calculating the optimal policy of such problems is computationally demanding and becomes intractable for large problems. Here, we address the problem of implementing several actions of different durations simultaneously. We demonstrate analytically that synchronising actions and their durations provide upper and lower bounds of the optimal performance. These bounds provide a simple way to evaluate the performance of any policy, including rules of thumb. We apply this approach to the management of a dynamic ecological network of Aedes albopictus, an invasive mosquito that vectors human diseases. The objective is to prevent mosquitoes from colonising mainland Australia from the nearby Torres Straits Islands where managers must decide between management actions that differ in duration and effectiveness. We were unable to compute an optimal policy for more than eight islands out of 17, but obtained upper and lower bounds for up to 13 islands. These bounds are within 16% of an optimal policy. We used the bounds to recommend managing highly populated islands as a priority. Our approach calculates upper and lower bounds for the optimal policy by solving simpler problems that are guaranteed to perform better and worse than the optimal policy respectively. By providing bounds on the optimal solution, the performance of policies can be evaluated even if the optimal policy cannot be calculated. Our general approach can be replicated for problems where simultaneous actions of different durations need to be implemented.

[1]  Carl J. Walters,et al.  ECOLOGICAL OPTIMIZATION AND ADAPTIVE MANAGEMENT , 1978 .

[2]  Carl J. Walters,et al.  Adaptive Management of Renewable Resources , 1986 .

[3]  Marc Mangel,et al.  Dynamic models in behavioural and evolutionary ecology , 1988, Nature.

[4]  W. Hawley The biology of Aedes albopictus. , 1988, Journal of the American Mosquito Control Association. Supplement.

[5]  Hawley Wa The biology of Aedes albopictus. , 1988 .

[6]  P. Ladiges,et al.  SEEDLING MORTALITY AND SUBMERGENCE IN DIPLACHNE FUSCA: A SEMI-AQUATIC WEED OF RICE FIELDS , 1989 .

[7]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[8]  Michael O. Duff,et al.  Reinforcement Learning Methods for Continuous-Time Markov Decision Problems , 1994, NIPS.

[9]  P. Phelan,et al.  Soil-Management History and Host Preference by Ostrinia nubilalis: Evidence for Plant Mineral Balance Mediating Insect–Plant Interactions , 1996 .

[10]  Satinder P. Singh,et al.  How to Dynamically Merge Markov Decision Processes , 1997, NIPS.

[11]  Ronen I. Brafman,et al.  Planning with Concurrent Interacting Actions , 1997, AAAI/IAAI.

[12]  Paul H. Williams,et al.  Biodiversity indicators: graphical techniques, smoothing and searching for what makes relationships work , 1998 .

[13]  Jesse Hoey,et al.  SPUDD: Stochastic Planning using Decision Diagrams , 1999, UAI.

[14]  A. Gill,et al.  Using stochastic dynamic programming to determine optimal fire management for Banksia ornata , 2001 .

[15]  Alessandro Vespignani,et al.  Epidemic spreading in scale-free networks. , 2000, Physical review letters.

[16]  Sridhar Mahadevan,et al.  Learning to Take Concurrent Actions , 2002, NIPS.

[17]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[18]  Sridhar Mahadevan,et al.  Recent Advances in Hierarchical Reinforcement Learning , 2003, Discret. Event Dyn. Syst..

[19]  James S. Clark,et al.  Why environmental scientists are becoming Bayesians , 2004 .

[20]  S. Ritchie,et al.  Aedes (Stegomyia) albopictus--a dengue threat for southern Australia? , 2005, Communicable diseases intelligence quarterly report.

[21]  Hugh P. Possingham,et al.  Prioritizing global conservation efforts , 2006, Nature.

[22]  Carolin Kreber,et al.  New Directions for Teaching and Learning , 2007 .

[23]  S. Ritchie,et al.  Discovery of a Widespread Infestation of Aedes albopictus in the Torres Strait, Australia , 2006, Journal of the American Mosquito Control Association.

[24]  François Charpillet,et al.  Dynamic optimization over infinite-time horizon: web-building strategy in an orb-weaving spider as a case study. , 2006, Journal of theoretical biology.

[25]  Thomas J. Walsh,et al.  Towards a Unified Theory of State Abstraction for MDPs , 2006, AI&M.

[26]  Cindy E. Hauser,et al.  Experimental or precautionary? Adaptive management over a range of time horizons , 2007 .

[27]  Tracy M. Rout,et al.  Managing beyond the invader: manipulating disturbance of natives simplifies control efforts , 2008 .

[28]  J. Pitt Modelling the spread of invasive species across heterogeneous landscapes , 2008 .

[29]  Brendan A. Wintle,et al.  When to stop managing or surveying cryptic threatened species , 2008, Proceedings of the National Academy of Sciences.

[30]  V. Bisaro,et al.  Individual and combined effects of Bacillus thuringiensis var. israelensis, temephos and Leptolegniachapmanii on the larval mortality of Aedes aegypti , 2010, BioControl.

[31]  I. Chades,et al.  Beyond stochastic dynamic programming: a heuristic sampling method for optimizing conservation decisions in very large state spaces , 2011 .

[32]  I. Chades,et al.  General rules for managing and surveying networks of pests, diseases, and endangered species , 2011, Proceedings of the National Academy of Sciences.

[33]  I. Chades,et al.  Optimally managing under imperfect detection: A method for plant invasions , 2011 .

[34]  Ljusk Ola Eriksson,et al.  Management of the risk of wind damage in forestry: a graph-based Markov decision process approach , 2011, Ann. Oper. Res..

[35]  Lieven Verschaffel,et al.  Heuristics and problem solving , 2012 .

[36]  I. Chades,et al.  Simple rules to contain an invasive species with a complex life cycle and high dispersal capacity , 2012 .

[37]  K. Mengersen,et al.  Eliciting Expert Knowledge in Conservation Science , 2012, Conservation biology : the journal of the Society for Conservation Biology.

[38]  Kent D. Messer,et al.  Cost-effective conservation planning: lessons from economics. , 2013, Journal of environmental management.

[39]  A. James,et al.  The invasive mosquito species Aedes albopictus: current knowledge and future perspectives. , 2013, Trends in parasitology.

[40]  S. Ritchie,et al.  Tracing the Tiger: Population Genetics Provides Valuable Insights into the Aedes (Stegomyia) albopictus Invasion of the Australasian Region , 2013, PLoS neglected tropical diseases.

[41]  Lucile Marescot,et al.  Complex decisions made simple: a primer on stochastic dynamic programming , 2013 .

[42]  A. Hoffmann,et al.  Predicting the spread of Aedes albopictus in Australia under current and future climates: Multiple approaches and datasets to incorporate potential evolutionary divergence , 2014 .

[43]  Hugh P. Possingham,et al.  A decision framework for management of conflicting production and biodiversity goals for a commercially valuable invasive species , 2014 .

[44]  I. Chades,et al.  Adapting environmental management to uncertain but inevitable change , 2015, Proceedings of the Royal Society B: Biological Sciences.

[45]  Arturo Carrillo-Reyes,et al.  State of knowledge and conservation of endangered and critically endangered lagomorphs worldwide , 2015 .

[46]  Gurutzeta Guillera-Arroita,et al.  When do we need more data? A primer on calculating the value of information for applied ecologists , 2015 .

[47]  C. Mantyka‐Pringle,et al.  Prioritizing management actions for the conservation of freshwater biodiversity under changing climate and land-cover , 2016 .

[48]  Supporting Information.docx , 2017 .

[49]  Simultaneous actions.rar , 2017 .