Markov decision processes in natural resources management: Observability and uncertainty

The breadth and complexity of stochastic decision processes in natural resources presents a challenge to analysts who need to understand and use these approaches. The objective of this paper is to describe a class of decision processes that are germane to natural resources conservation and management, namely Markov decision processes, and to discuss applications and computing algorithms under different conditions of observability and uncertainty. A number of important similarities are developed in the framing and evaluation of different decision processes, which can be useful in their applications in natural resources management. The challenges attendant to partial observability are highlighted, and possible approaches for dealing with it are discussed.

[1]  George E. Monahan,et al.  A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 2007 .

[2]  B. Sohngen,et al.  An Optimal Control Model of Forest Carbon Sequestration , 2003 .

[3]  Byron K. Williams,et al.  Uncertainty, learning, and the optimal management of wildlife , 2001, Environmental and Ecological Statistics.

[4]  J. D. Stigter,et al.  Optimal harvesting in a two-species model under critical depensation The case of optimal harvesting in semi-arid grazing systems , 2004 .

[5]  L. Joseph,et al.  Bayesian Statistics: An Introduction , 1989 .

[6]  Hugh P. Possingham,et al.  Optimal release strategies for biological control agents: an application of stochastic dynamic programming to population management , 2000 .

[7]  Wayne M. Getz,et al.  The use of stochastic dynamic programming in optimal landscape reconstruction for metapopulations , 2003 .

[8]  Warren B. Powell,et al.  “Approximate dynamic programming: Solving the curses of dimensionality” by Warren B. Powell , 2007, Wiley Series in Probability and Statistics.

[9]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[10]  Hugh P. Possingham,et al.  OPTIMAL FIRE MANAGEMENT FOR MAINTAINING COMMUNITY DIVERSITY , 1999 .

[11]  Carl J. Walters,et al.  TRADE-OFFS IN ECOSYSTEM-SCALE OPTIMIZATION OF FISHERIES MANAGEMENT POLICIES , 2004 .

[12]  Chelsea C. White,et al.  A survey of solution techniques for the partially observed Markov decision process , 1991, Ann. Oper. Res..

[13]  Carl J. Walters,et al.  Adaptive Management of Renewable Resources , 1986 .

[14]  James D. Nichols,et al.  Systems identification and the adaptive management of waterfowl in the United States , 2001, Wildlife Biology.

[15]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[16]  M. Littman,et al.  Efficient dynamic-programming updates in partially observable Markov decision processes , 1995 .

[17]  J. Nichols,et al.  Monitoring for conservation. , 2006, Trends in ecology & evolution.

[18]  B. K. Williams.  Optimal management strategies in variable environments: stochastic optimal control methods , 1985 .

[19]  Anne-Sophie Crépin,et al.  Multiple Species Boreal Forests – What Faustmann Missed , 2003 .

[20]  Sun Joseph Chang,et al.  A generalized Faustmann model for the determination of optimal harvest age , 1998 .

[21]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[22]  J SondikEdward.  The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon , 1978 .

[23]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[24]  James D. Nichols,et al.  Adaptive harvest management of North American waterfowl populations: a brief history and future prospects , 2007, Journal of Ornithology.

[25]  D. Newman,et al.  Forestry's golden rule and the development of the optimal forest rotation literature , 2002 .

[26]  M. Conroy,et al.  Analysis and Management of Animal Populations , 2002 .

[27]  Richard T. Roush,et al.  ACTIVE ADAPTIVE MANAGEMENT IN INSECT PEST AND WEED CONTROL: INTERVENTION WITH A PLAN FOR LEARNING , 2002 .

[28]  Michael D. Intriligator,et al.  Mathematical optimization and economic theory , 1971 .

[29]  Byron K. Williams,et al.  Review of dynamic optimization methods in renewable natural resource management , 1989 .

[30]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[31]  Hsien-Te Cheng,et al.  Algorithms for partially observable markov decision processes , 1989 .

[32]  Daniel E. Lane,et al.  A Partially Observable Model of Decision Making by Fishermen , 1989, Oper. Res..

[33]  F. Johnson,et al.  MANAGING NORTH AMERICAN WATERFOWL IN THE FACE OF UNCERTAINTY , 1995 .

[34]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over the Infinite Horizon: Discounted Costs , 1978, Oper. Res..

[35]  Duane Chapman,et al.  Timber harvest adjacency economies, hunting, species protection, and old growth value: seeking the dynamic optimum , 2003 .

[36]  Tyler Smith,et al.  Bayesian methods in hydrologic modeling: A study of recent advancements in Markov chain Monte Carlo techniques , 2008 .

[37]  Juan Guillermo Vaca-Rodríguez,et al.  Analysis of the eastern Pacific yellowfin tuna fishery based on multiple management objectives , 2006 .

[38]  C. Loehle.  Control theory and the management of ecosystems , 2006 .

[39]  C. Walters.  Challenges in adaptive management of riparian and coastal ecosystems , 1997 .