On the viscosity solutions of hybrid HJB equations arising in optimal control of loss networks

E § Abstract. Call admission and routing controls for loss (circuit-switched) networks with semi- Markovian, multi-class call arrivals and general connection durations, were formulated as optimal stochastic control problems in (12, 13). Each of the resulting so-called (network) hybrid HJB equations corresponds to a collection of coupled first-order partial differential equations for which, when it exists, the continuously differentiable value function is a solution to the associated hybrid HJB equations. In general, the smoothness of the value functions and uniqueness of the solutions to the hybrid HJB equations may not hold. In this paper, viscosity solutions to a general class of hybrid HJB equations are developed and under mild conditions it is shown that the value function is continuous and, further, any continuous value function is the unique viscosity solution to the hybrid HJB equations.

[1]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[2]  E. Altman,et al.  On submodular value functions and complex dynamic programming , 1998 .

[3]  Daniel P. Heyman,et al.  Optimal Operating Policies for M/G/1 Queuing Systems , 1968, Oper. Res..

[4]  N. Yamada,et al.  Perron ’ s Method for Viscosity Solutions Associated with Piecewise-Deterministic Processes , 2005 .

[5]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[6]  Peter E. Caines,et al.  Degenerate Stochastic Control Problems with ExponentialCosts and Weakly Coupled Dynamics: Viscosity Solutions and a Maximum Principle , 2005, SIAM J. Control. Optim..

[7]  S. Zachary,et al.  Loss networks , 2009, 0903.0640.

[8]  W. Fleming,et al.  Controlled Markov processes and viscosity solutions , 1992 .

[9]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[10]  Shaler Stidham,et al.  A survey of Markov decision models for control of networks of queues , 1993, Queueing Syst. Theory Appl..

[11]  Zhongjing Ma,et al.  Stochastic Control of Network Systems II: NETCAD Optimal Control & the HJB Equation , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[12]  Jane J. Ye.  Generalized Bellman-Hamilton-Jacobi equations for piecewise deterministic Markov processes , 1993, System Modelling and Optimization.

[13]  Zhongjing Ma,et al.  Stochastic Control of Network Systems I: NETCAD State Space Structure & Dynamics , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[14]  B. Hajek.  Optimal control of two interacting service stations , 1982, 1982 21st IEEE Conference on Decision and Control.

[15]  R. Newcomb.  VISCOSITY SOLUTIONS OF HAMILTON-JACOBI EQUATIONS , 2010 .

[16]  Peter E. Caines,et al.  Control of Loss Network Systems: Call Admission and Routing Control , 2007 .

[17]  Peter E. Caines,et al.  On the Viscosity Solutions of Hybrid HJB Equations Arising in Loss Network System Control , 2007 .

[18]  T. Başar,et al.  Optimal control of piecewise deterministic nonlinear systems with controlled transitions: viscosity solutions, their existence and uniqueness , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[19]  Peter E. Caines,et al.  Control of Admission and Routing in Loss Networks: Hybrid Dynamic Programming Equations , 2010, IEEE Transactions on Automatic Control.