Controlled Markov processes on the infinite planning horizon: Weighted and overtaking cost criteria

Stochastic control problems for controlled Markov processes models with an infinite planning horizon are considered, under some non-standard cost criteria. The classical discounted and average cost criteria can be viewed as complementary, in the sense that the former captures the short-time and the latter the long-time performance of the system. Thus, we study a cost criterion obtained as weighted combinations of these criteria, extending to a general state and control space framework several recent results by Feinberg and Shwartz, and by Krass et al. In addition, a functional characterization is given for overtaking optimal policies, for problems with countable state spaces and compact control spaces; our approach is based on qualitative properties of the optimality equation for problems with an average cost criterion.

[1]  Eugene A. Feinberg,et al.  Markov Decision Models with Weighted Discounted Criteria , 1994, Math. Oper. Res..

[2]  M. K. Ghosh,et al.  Discrete-time controlled Markov processes with average cost criterion: a survey , 1993 .

[3]  Manfred Schäl,et al.  Average Optimality in Dynamic Programming with General State Space , 1993, Math. Oper. Res..

[4]  Linn I. Sennott,et al.  Optimal Stationary Policies in General State Space Markov Decision Chains with Finite Action Sets , 1992, Math. Oper. Res..

[5]  Jerzy A. Filar,et al.  A Weighted Markov Decision Process , 1992, Oper. Res..

[6]  O. Hernández-Lerma,et al.  Equivalence of Lyapunov stability criteria in a class of Markov decision processes , 1992 .

[7]  Vivek S. Borkar,et al.  Ergodic and adaptive control of nearest-neighbor motions , 1991, Math. Control. Signals Syst..

[8]  O. J. Vrieze,et al.  Weighted reward criteria in Competitive Markov Decision Processes , 1989, ZOR Methods Model. Oper. Res..

[9]  A. Arapostathis,et al.  Remarks on the existence of solutions to the average cost optimality equation in Markov decision processes , 1991 .

[10]  V. Borkar Topics in controlled Markov chains , 1991 .

[11]  O. Hernández-Lerma,et al.  Average cost optimal policies for Markov control processes with Borel state space and unbounded costs , 1990 .

[12]  M. K. Ghosh Markov decision processes with multiple costs , 1990 .

[13]  O. Hernández-Lerma Adaptive Markov Control Processes , 1989 .

[14]  D. J. White,et al.  Further Real Applications of Markov Decision Processes , 1988 .

[15]  Arie Leizarowitz Controlled diffusion processes on infinite horizon with the overtaking criterion , 1988 .

[16]  Shaler Stidham,et al.  Scheduling, Routing, and Flow Control in Stochastic Networks , 1988 .

[17]  Arie Leizarwitz Infinite horizon optimization for finite state Markov chain , 1987 .

[18]  Dimitri P. Bertsekas,et al.  Dynamic Programming: Deterministic and Stochastic Models , 1987 .

[19]  Henk Tijms,et al.  Stochastic modelling and analysis: a computational approach , 1986 .

[20]  D. J. White,et al.  Real Applications of Markov Decision Processes , 1985 .

[21]  Vivek S. Borkar,et al.  Controlled Markov Chains and Stochastic Networks , 1983 .

[22]  E. Fainberg Controlled Markov Processes with Arbitrary Numerical Criteria , 1983 .

[23]  S. Yakowitz Dynamic programming applications in water resources , 1982 .

[24]  U. Rieder Measurable selection theorems for optimization problems , 1978 .

[25]  A. A. Yushkevich,et al.  On a Class of Strategies in General Markov Decision Models , 1974 .

[26]  K. Hinderer,et al.  Foundations of Non-stationary Dynamic Programming with Discrete Time Parameter , 1970 .

[27]  Cyrus Derman,et al.  Finite State Markovian Decision Processes , 1970 .

[28]  D. Gale On Optimal Development in a Multi-Sector Economy , 1967 .

[29]  C. Derman,et al.  A Note on Memoryless Rules for Controlling Sequential Control Processes , 1966 .

[30]  von Weizäscker,et al.  Existence of Optimal Programs of Accumulation for an Infinite Time Horizon , 1965 .

[31]  Onésimo Hernández-Lerma,et al.  Controlled Markov Processes , 1965 .