Ergodic Control of a Singularly Perturbed Markov Process in Discrete Time with General State and Compact Action Spaces

Abstract. Ergodic control of singularly perturbed Markov chains with general state and compact action spaces is considered. A new method is given for characterization of the limit of invariant measures, for perturbed chains, when the perturbation parameter goes to zero. It is also demonstrated that the limit control principle is satisfied under natural ergodicity assumptions about controlled Markov chains. These assumptions allow for the presence of transient states, a situation that has not been considered in the literature before in the context of control of singularly perturbed Markov processes with long-run-average cost functionals.

[1]  Eitan Altman,et al.  Control of a hybrid stochastic system , 1993 .

[2]  H. Kushner.  Weak Convergence Methods and Singularly Perturbed Stochastic Control and Filtering Problems , 1990 .

[3]  P. WHITTLE,et al.  Markov Processes and Their Applications , 1962, Nature.

[4]  Jerzy A. Filar,et al.  Singularly perturbed Markov control problem: Limiting average cost , 1991 .

[5]  Alain Bensoussan,et al.  Singular perturbations in stochastic control , 1984 .

[6]  V. G. Gaitsgori,et al.  Theory of Suboptimal Decisions , 1988 .

[7]  Łukasz Stettner,et al.  On ergodic control problems for singularly perturbed Markov processes , 1989 .

[8]  Qing Zhang,et al.  Hierarchical Decision Making in Stochastic Manufacturing Systems , 1994 .

[9]  Łukasz Stettner,et al.  On nearly self-optimizing strategies for a discrete-time uniformly ergodic adaptive model , 1993 .

[10]  T. Bielecki,et al.  On some problems arising in asymptotic analysis of Markov processes with singularly perturbed generators , 1988 .

[11]  Onésimo Hernández-Lerma,et al.  Controlled Markov Processes , 1965 .

[12]  François Delebecque,et al.  Optimal control of markov chains admitting strong and weak interactions , 1981, Autom..

[13]  J. Filar,et al.  Algorithms for singularly perturbed limiting average Markov control problems , 1992 .

[14]  P. Kokotovic,et al.  Singular perturbations and asymptotic analysis in control systems , 1986 .

[15]  J. Neveu.  Bases mathématiques du calcul des probabilités , 1966 .

[16]  A. Bensoussan.  Perturbation Methods in Optimal Control , 1988 .

[17]  Sheldon M. Ross,et al.  Stochastic Processes , 2018, Gauge Integral Structures for Stochastic Calculus and Quantum Electrodynamics.