Ergodic control of Markov chains

The ergodic or long-run average cost problem, one of the classical problems in Markov decision processes, is addressed. A brief description of Borkar's convex analytic approach is presented which treats the problem with countable state space, compact action space, and unbounded cost. The ergodic control problem is related to the finite horizon and discounted cost problems.<<ETX>>

[1]  V. Borkar.  Control of Markov chains with long-run average cost criterion: the dynamic programming equations , 1989 .

[2]  C. Derman.  DENUMERABLE STATE MARKOVIAN DECISION PROCESSES: AVERAGE COST CRITERION. , 1966 .

[3]  R. Cavazos-Cadena.  Necessary conditions for the optimality equation in average-reward Markov decision processes , 1989 .

[4]  H. M. Taylor.  Markovian sequential replacement processes , 1965 .

[5]  C. Derman,et al.  A SOLUTION TO A COUNTABLE SYSTEM OF EQUATIONS ARISING IN MARKOVIAN DECISION PROCESSES. , 1966 .

[6]  Linn I. Sennott,et al.  Average Cost Optimal Stationary Policies in Infinite State Markov Decision Processes with Unbounded Costs , 1989, Oper. Res..

[7]  A. Arapostathis,et al.  On partially observable Markov decision processes with an average cost criterion , 1989, Proceedings of the 28th IEEE Conference on Decision and Control,.

[8]  C. Derman.  On Sequential Decisions and Markov Chains , 1962 .

[9]  Zvi Rosberg,et al.  Optimal control of service in tandem queues , 1982 .

[10]  Arie Hordijk,et al.  Dynamic programming and Markov potential theory , 1974 .

[11]  S. Ross.  Arbitrary State Markovian Decision Processes , 1968 .

[12]  James Flynn.  Averaging vs. Discounting in Dynamic Programming: a Counterexample , 1974 .

[13]  V. Borkar.  On Minimum Cost Per Unit Time Control of Markov Chains , 1984 .

[14]  R. Cavazos-Cadena.  Necessary and sufficient conditions for a bounded solution to the optimality equation in average reward Markov decision chains , 1988 .

[15]  Vivek S. Borkar,et al.  Control of Markov Chains with Long-Run Average Cost Criterion , 1988 .

[16]  Paul J. Schweitzer,et al.  Denumerable Undiscounted Semi-Markov Decision Processes with Unbounded Rewards , 1983, Math. Oper. Res..

[17]  R. Weber,et al.  Optimal control of service rates in networks of queues , 1987, Advances in Applied Probability.

[18]  Linn I. Sennott,et al.  Average Cost Semi-Markov Decision Processes and the Control of Queueing Systems , 1989, Probability in the Engineering and Informational Sciences.

[19]  Ari Arapostathis,et al.  On the average cost optimality equation and the structure of optimal policies for partially observable Markov decision processes , 1991, Ann. Oper. Res..

[20]  James Flynn.  On optimality criteria for dynamic programs with long finite horizons , 1980 .

[21]  D. Blackwell.  Discrete Dynamic Programming , 1962 .

[22]  Shaler Stidham,et al.  Monotonic and Insensitive Optimal Policies for Control of Queues with Undiscounted Costs , 1989, Oper. Res..