A Minimum Principle for Controlled Jump Processes

In Queueing Theory and many other fields problems of control arise for stochastic processes with piecewise constant paths. In this paper the validity of optimality conditions analagous to the Pontryagin Maximum Principle for deterministic control problems is investigated for this type of stochastic process. A minimum principle which involves the conditional jump rate, the conditional state jump distribution, system performance rate, and the conditional expectation of the remaining performance is obtained. The conditional expectation of the remaining performance plays the role of the adjoint variables. This conditional expectation satisfies a type of integral equation and an infinite system of ordinary differential equations.