Linear programming versions of some control problems on Markov chains are derived, and are studied under conditions which occur in typical problems which arise by discretizing continuous time and state systems or in discrete state systems. Control interpretations of the dual variables and simplex multipliers are given. The formulations allows the treatment of ‘state space’ like constraints which cannot be handled conveniently with dynamic programming. The relation between dynamic programming on Markov chains and the deterministic discrete maximum principle is explored, and some insight is obtained into the problem of singular stochastic controls (with respect to a stochastic maximum principle).
[1]
G. Dantzig,et al.
Linear Programming in a Markov Chain
,
1962
.
[2]
George B. Dantzig,et al.
Linear programming and extensions
,
1965
.
[3]
H. Kushner.
On the stochastic maximum principle with “average” constraints☆
,
1965
.
[4]
H. Kushner,et al.
Numerical methods for the solution of the degenerate nonlinear elliptic equations arising in optimal stochastic control theory
,
1968
.
[5]
E. Polak,et al.
Theory of optimal control and mathematical programming
,
1969
.
[6]
Patrick Rivett,et al.
Principles of Operations Research
,
1972
.