Playing games against nature: optimal policies for renewable resource allocation

In this paper we introduce a class of Markov decision processes that arise as a natural model for many renewable resource allocation problems. Upon extending results from the inventory control literature, we prove that they admit a closed form solution and we show how to exploit this structure to speed up its computation. We consider the application of the proposed framework to several problems arising in very different domains, and as part of the ongoing effort in the emerging field of Computational Sustainability we discuss in detail its application to the Northern Pacific Halibut marine fishery. Our approach is applied to a model based on real world data, obtaining a policy with a guaranteed lower bound on the utility function that is structurally very different from the one currently employed.

[1]  L Doyen,et al.  Sustainability of fisheries through marine reserves: a robust modeling analysis. , 2003, Journal of environmental management.

[2]  Arkadi Nemirovski,et al.  Robust optimization – methodology and applications , 2002, Math. Program..

[3]  William J. Reed,et al.  A stochastic model for the economic management of a renewable animal resource , 1974 .

[4]  H. Scarf.  THE OPTIMALITY OF (S,S) POLICIES IN THE DYNAMIC INVENTORY PROBLEM , 1959 .

[5]  Michael L. Littman,et al.  Markov Games as a Framework for Multi-Agent Reinforcement Learning , 1994, ICML.

[6]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[7]  Melvyn Sim,et al.  Robust discrete optimization and network flows , 2003, Math. Program..

[8]  Daniel Hernández-Hernández,et al.  Risk Sensitive Markov Decision Processes , 1997 .

[9]  M. Dufwenberg.  Game theory. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[10]  Marc Mangel,et al.  IMPLEMENTING THE PRECAUTIONARY PRINCIPLE IN FISHERIES MANAGEMENT THROUGH MARINE RESERVES , 1998 .

[11]  C. Gomes.  Computational Sustainability: Computational methods for a sustainable environment, economy, and society , 2009 .

[12]  Colin W. Clark,et al.  Mathematical Bioeconomics: The Optimal Management of Renewable Resources. , 1993 .

[13]  William J. Reed,et al.  Optimal escapement levels in stochastic and deterministic harvesting models , 1979 .

[14]  Frances R. Homans,et al.  A Model of Regulated Open Access Resource Use , 1997 .