Singular control with state constraints on unbounded domain

We study a class of stochastic control problems where a cost of the form E ∫ [0,∞) e -βs [l(X s )ds + h(Y o s )d|Y| s ] is to be minimized over control processes Y whose increments take values in a cone Y of R p , keeping the state process X = x + B + GY in a cone X of R k , k < p. Here, x e X, B is a Brownian motion with drift b and covariance E, G is a fixed matrix, and Y° is the Radon-Nikodym derivative dY/d|Y|. Let £ = -(1/2)trace(ΣD 2 ) - b · D where D denotes the gradient. Solutions to the corresponding dynamic programming PDE, [(£+β)f-l]V sup [-Gγ·Df-h(γ)]=0, γ∈Y:|Gγ|=1 on X" are considered with a polynomial growth condition and are required to be supersolution up to the boundary (corresponding to a "state constraint" boundary condition on 3X). Under suitable conditions on the problem data, including continuity and nonnegativity of l and h, and polynomial growth of l, our main result is the unique viscosity-sense solvability of the PDE by the control problem's value function in appropriate classes of functions. In some cases where uniqueness generally fails to hold in the class of functions that grow at most polynomially (e.g., when h = 0), our methods provide uniqueness within the class of functions that, in addition, have compact level sets. The results are new even in the following special cases: (1) The one-dimensional case k = p = 1, X = Y = R + ; (2) The first-order case Σ = 0; (3) The case where l and h are linear. The proofs combine probabilistic arguments and viscosity solution methods. Our framework covers a wide range of diffusion control problems that arise from queueing networks in heavy traffic.

[1]  E. Wong Representation of Martingales, Quadratic Variation and Applications , 1971 .

[2]  Volker Bã–hm On the continuity of the optimal policy set for linear programs , 1975 .

[3]  Volker Böhm On the continuity of the optimal policy set for linear programs , 1975 .

[4]  丸山 徹 Convex Analysisの二,三の進展について , 1977 .

[5]  Lawrence C. Evans,et al.  A second order elliptic equation with gradient constraint , 1979 .

[6]  D. W. Stroock,et al.  Multidimensional Diffusion Processes , 1979 .

[7]  H. Witsenhausen,et al.  Some solvable stochastic control problemst , 1980 .

[8]  P. Lions Generalized Solutions of Hamilton-Jacobi Equations , 1982 .

[9]  Maurice Robin,et al.  On some cheap control problems for diffusion processes , 1983 .

[10]  I. Karatzas A class of singular stochastic control problems , 1983, Advances in Applied Probability.

[11]  J. Michael Harrison,et al.  Instantaneous Control of Brownian Motion , 1983, Math. Oper. Res..

[12]  Hitoshi Ishii,et al.  Boundary regulatity and uniqueness for an elliptic equations with gradient constraint , 1983 .

[13]  Maurice Robin,et al.  Additive control of stochastic linear systems with finite horizon , 1984, The 23rd IEEE Conference on Decision and Control.

[14]  H. Soner Optimal control with state-space constraint I , 1986 .

[15]  Pierre-Louis Lions,et al.  Remarks on the Existence and Uniqueness of Unbounded Viscosity Solutions of Hamilton-Jacobi Equations. , 1987 .

[16]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[17]  J. Michael Harrison,et al.  Brownian Models of Queueing Networks with Heterogeneous Customer Populations , 1988 .

[18]  H. Ishii On uniqueness and existence of viscosity solutions of fully nonlinear second‐order elliptic PDE's , 1989 .

[19]  P. Lions,et al.  Nonlinear elliptic equations with singular boundary conditions and stochastic control with state constraints , 1989 .

[20]  Vivek S. Borkar,et al.  Optimal Control of Diffusion Processes , 1989 .

[21]  R. Durrett Probability: Theory and Examples , 1993 .

[22]  P. Lions,et al.  Hamilton-Jacobi equations with state constraints , 1990 .

[23]  M. Yor,et al.  Representation of Martingales , 1991 .

[24]  P. Dupuis,et al.  On Lipschitz continuity of the solution mapping to the Skorokhod problem , 1991 .

[25]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[26]  P. Lions,et al.  User’s guide to viscosity solutions of second order partial differential equations , 1992, math/9207212.

[27]  Michael G. Crandall,et al.  Viscosity Solutions of Fully Nonlinear Equations , 1994 .

[28]  H. Soner,et al.  Optimal Investment and Consumption with Transaction Costs , 1994 .

[29]  L. F. Martins,et al.  Heavy Traffic Convergence of a Controlled, Multiclass Queueing System , 1996 .

[30]  M. Bardi,et al.  Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations , 1997 .

[31]  Jan A. Van Mieghem,et al.  Dynamic Control of Brownian Networks: State Space Collapse and Equivalent Workload Formulations , 1997 .

[32]  W. Fleming,et al.  Hedging in incomplete markets with HARA utility , 1997 .

[33]  P. Dupuis,et al.  Convex duality and the Skorokhod Problem. II , 1999 .

[34]  P. Dupuis,et al.  Convex duality and the Skorokhod Problem. I , 1999 .

[35]  J. Harrison Brownian models of open processing networks: canonical representation of workload , 2000 .

[36]  Paola Loreti,et al.  A class of stochastic optimal control problems with state constraint , 2002 .

[37]  Maury Bramson,et al.  Two Workload Properties for Brownian Networks , 2003, Queueing Syst. Theory Appl..