Hierarchic Markov processes and their applications in replacement models

Abstract In this paper a new notion of a hierarchic Markov process is introduced. It is a series of Markov decision processes called subprocesses built together in one Markov decision process called the main process. The hierarchic structure is specially designed to fit replacement models which in the traditional formulation as ordinary Markov decision processes are usually very large. The basic theory of hierarchic Markov processes is described and examples are given of applications in replacement models. The theory can be extended to fit a situation where the replacement decision depends on the quality of the new asset available for replacement.

[1]  Moshe Haviv,et al.  Truncated policy iteration methods , 1984 .

[2]  George Willard Terborgh,et al.  Dynamic equipment policy , 1949 .

[3]  J. A. Arendonk,et al.  Studies on the replacement policies in dairy cattle. II. Optimum policy and influence of changes in production and prices , 1985 .

[4]  Gabriel A. D. Preinreich.  The Economic Life of Industrial Equipment , 1940 .

[5]  McArthur Atg,et al.  Application of Dynamic Programming to the Culling Decision in Dairy Cattle , 1973 .

[6]  Aalt A. Dijkhuizen,et al.  Studies on the replacement policies in dairy cattle. III. Influence of variation in reproduction and production , 1985 .

[7]  Birger Rapp.  Models for optimal investment and maintenance decisions , 1974 .

[8]  J Jaap Wessels,et al.  Markov decision processes , 1985 .

[9]  A. R. Kristensen.  A Model to Describe and Predict the Production and Food Intake of the Dairy Cow , 1986 .

[10]  Yaron Ben-Ari,et al.  Optimal replacement policy for multicomponent systems: An application to a dairy herd , 1986 .

[11]  R. Bellman.  Dynamic programming. , 1957, Science.

[12]  J. A. Arendonk,et al.  Studies on the replacement policies in dairy cattle IV. Influence of seasonal variation in performance and prices. , 1986 .

[13]  Ronald A. Howard,et al.  Dynamic Programming and Markov Processes , 1960 .

[14]  Anders Ringgaard Kristensen.  Optimal replacement and ranking of dairy cows determined by a hierarchic Markov process , 1987 .

[15]  E. B. Burnside,et al.  Optimal Culling Strategies for Dairy Cows of Different Breeds , 1978 .

[16]  E. B. Burnside,et al.  A Dynamic Programming Approach to Culling Decisions in Commercial Dairy Herds , 1977 .