Algorithms for Singularly Perturbed Markov Control Problems: A Survey11We are indebted to M. Haviv for a number of helpful discussionsand for his comments on anearlier draft of the manuscript.

[1]  H. Khalil,et al.  Aggregation of the policy iteration method for nearly completely decomposable Markov chains , 1991 .

[2]  Peter W. Jones,et al.  Stochastic Modelling and Analysis , 1988 .

[3]  Tosio Kato.  Perturbation theory for linear operators , 1966 .

[4]  M. Haviv.  Block-successive approximation for a discounted Markov decision model , 1985 .

[5]  P. Kokotovic,et al.  A singular perturbation approach to modeling and control of Markov chains , 1981 .

[6]  F. Delebecque.  A Reduction Process for Perturbed Markov Chains , 1983 .

[7]  J. Filar,et al.  Perturbation and stability theory for Markov control problems , 1992 .

[8]  S. Sastry,et al.  Hierarchical aggregation of singularly perturbed finite state Markov processes , 1983 .

[9]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[10]  Stephen P. Brooks,et al.  Markov Decision Processes. , 1995 .

[11]  O. Hernández-Lerma.  Adaptive Markov Control Processes , 1989 .

[12]  V. G. Gaitsgori,et al.  Theory of Suboptimal Decisions , 1988 .

[13]  J. Craggs.  Applied Mathematical Sciences , 1973 .

[14]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[15]  J. Doob.  Stochastic processes , 1953 .

[16]  J. Quadrat,et al.  Contribution of stochastic control singular perturbation averaging and team theories to an example of large-scale systems: Management of hydropower production , 1978 .

[17]  J. Filar,et al.  Algorithms for singularly perturbed limiting average Markov control problems , 1992 .

[18]  P. Schweitzer.  Perturbation theory and finite Markov chains , 1968 .

[19]  M. Puterman,et al.  Perturbation theory for Markov reward processes with applications to queueing systems , 1988, Advances in Applied Probability.

[20]  John G. Kemeny,et al.  Finite Markov Chains. , 1960 .

[21]  P J Schweitzer,et al.  Perturbation series expansions for nearly completely-decomposable Markov chains , 1986 .

[22]  L. C. M. Kallenberg,et al.  Linear programming and finite Markovian control problems , 1984 .

[23]  E. Altman,et al.  Stability and singular perturbations in constrained Markov decision problems , 1993, IEEE Trans. Autom. Control..

[24]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[25]  Eitan Altman,et al.  Sensitivity of constrained Markov decision processes , 1991, Ann. Oper. Res..

[26]  Nico M. van Dijk.  PERTURBATION THEORY FOR UNBOUNDED MARKOV REWARD PROCESSES WITH APPLICATIONS TO QUEUEING , 1988 .

[27]  Jerzy A. Filar,et al.  Singularly perturbed Markov control problem: Limiting average cost , 1991 .

[28]  D. Blackwell.  Discrete Dynamic Programming , 1962 .

[29]  Herbert A. Simon,et al.  Aggregation of Variables in Dynamic Systems , 1961 .

[30]  Cyrus Derman,et al.  Finite State Markovian Decision Processes , 1970 .

[31]  Refael Hassin,et al.  Mean Passage Times and Nearly Uncoupled Markov Chains , 1992, SIAM J. Discret. Math..

[32]  Jean B. Lasserre.  A formula for singular perturbations of Markov chains , 1994 .

[33]  Mashe Sniedovich,et al.  Dynamic Programming , 1991 .

[34]  François Delebecque,et al.  Optimal control of markov chains admitting strong and weak interactions , 1981, Autom..