Monitoring as a partially observable decision problem

Monitoring is an important and costly activity in resource management problems such as containing invasive species, protecting endangered species, preventing soil erosion, and regulating contracts for environmental services. Recent studies have viewed optimal monitoring as a Partially Observable Markov Decision Process (POMDP), which provides a framework for sequential decision making under stochastic resource dynamics and uncertainty about the resource state. We present an overview of the POMDP framework and its applications to resource monitoring. We discuss the concept of the information content provided by monitoring systems and illustrate how information content affects optimal monitoring strategies. Finally, we demonstrate how the timing of monitoring in relation to resource treatment and transition can have substantial effects on optimal monitoring strategies.

[1]  Daniel E. Lane,et al.  A Partially Observable Model of Decision Making by Fishermen , 1989, Oper. Res..

[2]  I. Chades,et al.  When to stop managing or surveying cryptic threatened species , 2008, Proceedings of the National Academy of Sciences.

[3]  Edward J. Sondik,et al.  The optimal control of par-tially observable Markov processes , 1971 .

[4]  Krishna Pacifici,et al.  Addressing structural and observational uncertainty in resource management. , 2014, Journal of environmental management.

[5]  Nikos A. Vlassis,et al.  Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[6]  B. White.  An economic analysis of ecological monitoring , 2005 .

[7]  J. Riley,et al.  The analytics of uncertainty and information: Long-run relationships and the credibility of threats and promises , 1992 .

[8]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[9]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[10]  Shobha Venkataraman,et al.  Efficient Solution Algorithms for Factored MDPs , 2003, J. Artif. Intell. Res..

[11]  Joelle Pineau,et al.  Anytime Point-Based Approximations for Large POMDPs , 2006, J. Artif. Intell. Res..

[12]  I. Chades,et al.  Optimally managing under imperfect detection: A method for plant invasions , 2011 .

[13]  Peter W J Baxter,et al.  Optimal eradication: when to stop looking for an invasive plant. , 2006, Ecology letters.

[14]  B. Williams.  Adaptive management of natural resources--framework and issues. , 2011, Journal of environmental management.

[15]  J. Nichols,et al.  Monitoring for conservation. , 2006, Trends in ecology & evolution.

[16]  Colin W. Clark,et al.  On uncertain renewable resource stocks: Optimal harvest policies and the value of stock surveys , 1986 .

[17]  David Hsu,et al.  SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces , 2008, Robotics: Science and Systems.

[18]  I. Chades,et al.  Allocating conservation resources between areas where persistence of a species is uncertain. , 2011, Ecological applications : a publication of the Ecological Society of America.

[19]  Alana L. Moore.  Managing populations in the face of uncertainty: adaptive management, partial observability and the dynamic value of information , 2008 .

[20]  W. Hanemann,et al.  Fishery Management Under Multiple Uncertainty , 2004 .

[21]  Erling Moxnes,et al.  Uncertain measurements of renewable resources: approximations, harvesting policies and value of accuracy , 2003 .

[22]  Reid G. Simmons,et al.  Point-Based POMDP Algorithms: Improved Analysis and Implementation , 2005, UAI.

[23]  D. Tomberlin,et al.  Endangered Seabird Habitat Management as a Partially Observable Markov Decision Process , 2010, Marine Resource Economics.

[24]  Leslie Pack Kaelbling,et al.  Planning and Acting in Partially Observable Stochastic Domains , 1998, Artif. Intell..

[25]  J. Marschak,et al.  ECONOMIC COMPARABILITY OF INFORMATION SYSTEMS. , 1968 .

[26]  George E. Monahan,et al.  A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 2007 .

[27]  Anthony R. Cassandra,et al.  Optimal Policies for Partially Observable Markov Decision Processes , 1994 .

[28]  David Hsu,et al.  POMDPs for robotic tasks with mixed observability , 2009, Robotics: Science and Systems.

[29]  C. Walters,et al.  Adaptive Control of Fishing Systems , 1976 .

[30]  Guy Shani,et al.  Forward Search Value Iteration for POMDPs , 2007, IJCAI.

[31]  H. Possingham,et al.  Monitoring does not always count. , 2010, Trends in ecology & evolution.

[32]  B. White,et al.  A Bioeconomic Analysis of the Duration of Conservation Contracts , 2007 .

[33]  Eve McDonald-Madden,et al.  Managing and learning with multiple models: Objectives and optimization algorithms , 2011 .

[34]  Olivier Buffet,et al.  MOMDPs: A Solution for Modelling Adaptive Management Problems , 2012, AAAI.

[35]  William S. Lovejoy,et al.  Computationally Feasible Bounds for Partially Observed Markov Decision Processes , 1991, Oper. Res..

[36]  Robert G. Haight,et al.  Optimal control of an invasive species with imperfect information about the level of infestation , 2010 .