Optimal Control under a Dynamic Fuel Constraint

We present a new approach to solve optimal control problems of the monotone follower type. The key feature of our approach is that it allows us to include an arbitrary dynamic fuel constraint. Instead of dynamic programming, we use the convexity of our cost functional to derive a first order characterization of optimal policies based on the Snell envelope of the objective functional's gradient at the optimum. The optimal control policy is constructed explicitly in terms of the solution to a representation theorem for stochastic processes obtained in Bank and El Karoui (2004), {Ann. Probab.}, 32, pp. 1030--1067. As an illustration, we show how our methodology allows us to extend the scope of the explicit solutions obtained for the classical monotone follower problem and for an irreversible investment problem arising in economics.

[1]  I Karatzas,et al.  Probabilistic aspects of finite-fuel stochastic control. , 1985, Proceedings of the National Academy of Sciences of the United States of America.

[2]  S. Jacka.  Avoiding the origin: A finite-fuel stochastic control problem , 2002 .

[3]  M. Kohlmann,et al.  Connections between optimal stopping and singular stochastic control , 1998 .

[4]  Ulrich G. Haussmann,et al.  The optimal control of the cheap monotone follower , 1994 .

[5]  S. Shreve,et al.  Connections Between Optimal Stopping and Singular Stochastic Control II. Reflected Follower Problems , 1985 .

[6]  Maria B. Chiarolla.  Singular stochastic control of a singular diffusion process , 1997 .

[7]  Frank Riedel,et al.  Optimal consumption choice with intertemporal substitution , 2001 .

[8]  Ioannis Karatzas,et al.  Irreversible investment and industry equilibrium , 1996, Finance Stochastics.

[9]  Maurice Robin,et al.  Additive control of stochastic linear systems with finite horizon , 1984, The 23rd IEEE Conference on Decision and Control.

[10]  N. Karoui,et al.  A stochastic representation theorem with applications to optimization and obstacle problems , 2004 .

[11]  S. Jacka.  Keeping a satellite aloft: two finite fuel stochastic control models , 1999, Journal of Applied Probability.

[12]  H. Witsenhausen,et al.  Some solvable stochastic control problemst , 1980 .

[13]  I. Karatzas,et al.  Probabilistic aspects of finite-fuel, reflected follower problems , 1988 .

[14]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[15]  I. Karatzas,et al.  A new approach to the skorohod problem, and its applications , 1991 .

[16]  Thaleia Zariphopoulou,et al.  Optimal Environmental Management in the Presence of Irreversibilities , 2001, J. Econ. Theory.

[17]  S. Shreve,et al.  Connections between Optimal Stopping and Singular Stochastic Control I. Monotone Follower Problems , 1984 .

[18]  T. Kobila.  A Class of Solvable Stochastic Investment Problems Involving Singular Controls , 1993 .

[19]  H. Föllmer,et al.  American Options, Multi–armed Bandits, and Optimal Consumption Plans: A Unifying View , 2003 .

[20]  Frank Riedel,et al.  On Irreversible Investment , 2006 .