Long-term average cost control problems for continuous time Markov processes: A survey

This paper addresses the long-term average cost control of continuous time Markov processes. A survey of problems and methods contained in various works is given for continuous control, optimal stopping, and impulse control.

[1]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[2]  D. Blackwell.  Discrete Dynamic Programming , 1962 .

[3]  Onésimo Hernández-Lerma,et al.  Controlled Markov Processes , 1965 .

[4]  E. Lanery,et al.  Étude asymptotique des systèmes markoviens à commande , 1967 .

[5]  Y. Kogan.  On Optimal Control of a Non-Terminating Diffusion Process with Reflection , 1969 .

[6]  S. Ross.  Average cost semi-markov decision processes , 1970, Journal of Applied Probability.

[7]  Sheldon M. Ross.  On the Nonexistence of $|varepsilon$-Optimal Randomized Stationary Policies in Average Cost Markov Decision Models , 1971 .

[8]  Steven A. Lippman.  Maximal Average-Reward Policies for Semi-Markov Decision Processes With Arbitrary State and Action Space , 1971 .

[9]  S. Varadhan,et al.  Diffusion processes with boundary conditions , 1971 .

[10]  P. Kakumanu,et al.  Nondiscounted Continuous Time Markovian Decision Process with Countable State Space , 1972 .

[11]  Arie Hordijk,et al.  Dynamic programming and Markov potential theory , 1974 .

[12]  Bharat T. Doshi,et al.  Continuous time control of Markov processes on an arbitrary state space: Average return criterion , 1976 .

[13]  Jacob Wijngaard,et al.  Stationary Markovian Decision Problems and Perturbation Theory of Quasi-Compact Linear Operators , 1977, Math. Oper. Res..

[14]  J. Bismut,et al.  Temps d'arrÊt optimal, théorie générale des processus et processus de Markov , 1977 .

[15]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Optimal stopping rules , 1977 .

[16]  A. Federgruen,et al.  Denumerable state semi-markov decision processes with unbounded costs, average cost criterion : (preprint) , 1979 .

[17]  M. Robin,et al.  Contrôle impulsionnel des processus de Markov , 1978 .

[18]  Michael Kohlmann,et al.  Stochastic Control Theory and Stochastic Differential Systems , 1979 .

[19]  B. Heimann,et al.  Fleming, W. H./Rishel, R. W., Deterministic and Stochastic Optimal Control. New York‐Heidelberg‐Berlin. Springer‐Verlag. 1975. XIII, 222 S, DM 60,60 , 1979 .

[20]  J. Menaldi.  On the Optimal Stopping Time Problem for Degenerate Diffusions , 1980 .

[21]  R. Khasminskii.  Stochastic Stability of Differential Equations , 1980 .

[22]  B. L. Miller.  Countable-state average-cost regenerative stopping problems , 1981, Journal of Applied Probability.

[23]  Massimo Lorenzani.  Optimal stopping for Markov Processes , 1981 .

[24]  M. Robin.  On Some Impulse Control Problems with Long Run Average Cost , 1981 .

[25]  A. Bensoussan.  Stochastic control by functional analysis methods , 1982 .

[26]  Zvi Rosberg,et al.  Optimal control of service in tandem queues , 1982 .

[27]  A. Bensoussan,et al.  Contrôle impulsionnel et inéquations quasi variationnelles , 1982 .

[28]  F. Kozin,et al.  System Modeling and Optimization , 1982 .

[29]  A. A. Yushkevich,et al.  On Semi-Markov Controlled Models with an Average Reward Criterion , 1982 .

[30]  J. Menaldi.  On a degenerate variational inequality with Neumann boundary conditions , 1982 .

[31]  Armind M. Makowski.  Local optimality conditions for optimal stopping , 1982 .

[32]  I. Karatzas.  A class of singular stochastic control problems , 1983, Advances in Applied Probability.

[33]  Asymptotics in quasi-variational inequalities and ergodic control problems , 1983 .

[34]  Łukasz Stettner,et al.  On impulsive control with long run average cost criterion , 1983 .