Economic Rewards in Non-homogeneous Semi-Markov Systems

Abstract In this paper, a general reward model for a non homogeneous semi-Markov system will be provided. Both discrete and continuous time models will be studied considering reward to be a random variable of the economic type associated with the state occupancies and transitions. Basic equations will be exhibited and the main formulas for the expected reward that the system will generate in some interval of time (with and without discounting) will be determined for the discrete and continuous cases and a method is provided in order to find it in closed analytic form in relation with the basic parameters of the system. Also, results will be obtained for the expected reward per time period for the steady state.

[1]  Erhan Cinlar,et al.  Markov Renewal Theory: A Survey , 1973 .

[2]  Sally McClean,et al.  Continuous-time stochastic models of a multigrade population , 1978, Journal of Applied Probability.

[3]  M. Neuts,et al.  The Integral of a Step Function Defined on a Semi-Markov Process , 1967 .

[4]  Sally McClean,et al.  Using a Markov reward model to estimate spend-down costs for a geriatric department , 1998, J. Oper. Res. Soc..

[5]  J. Keilson,et al.  A process with chain dependent growth rate , 1970, Journal of Applied Probability.

[6]  Jozef L. Teugels,et al.  A bibliography on semi-Markov processes , 1976 .

[7]  J. Keilson,et al.  On the Matrix Renewal Function for Markov Renewal Processes , 1969 .

[8]  Ronald Pyke,et al.  Limit Theorems for Markov Renewal Processes , 1964 .

[9]  E. Çinlar.  Markov renewal theory , 1969, Advances in Applied Probability.

[10]  Jacques Janssen,et al.  Finite non-homogeneous semi-Markov processes: Theoretical and computational aspects , 1984 .

[11]  Sally McClean,et al.  Semi-Markov models for manpower planning , 1986 .

[12]  Jacques Janssen.  Semi-Markov Models , 1986 .

[13]  Sally McClean,et al.  The Two‐Stage Model of Personnel Behaviour , 1976 .

[14]  Sally McClean.  A semi-Markov model for a multigrade population with Poisson recruitment , 1980 .

[15]  E. Çinlar.  Exceptional Paper---Markov Renewal Theory: A Survey , 1975 .

[16]  A. A. Papadopoulou.  Counting transitions—entrance probabilities in non-homogeneous semi-Markov systems , 1997 .

[17]  P.-C. G. Vassiliou,et al.  Non-homogeneous semi-Markov systems and maintainability of the state sizes , 1992 .

[18]  J. Keilson,et al.  A PROCESS WITH CHAIN DEPENDENT GROWTH RATE. PART II: THE RUIN AND ERGODIC PROBLEMS , 1971 .

[19]  P.-C. G. Vassiliou,et al.  Asymptotic behavior of nonhomogeneous semi-Markov systems , 1994 .

[20]  P. Vassiliou,et al.  Continuous Time Non Homogeneous Semi-Markov Systems , 1999 .

[21]  Sally McClean,et al.  Stochastic models of geriatric patient bed occupancy behaviour , 2000 .

[22]  Ronald A. Howard,et al.  Dynamic Probabilistic Systems , 1971 .

[23]  A. Soltani,et al.  Reward processes for semi-Markov processes: asymptotic behaviour , 1998, Journal of Applied Probability.