A Survey of Applications of Markov Decision Processes

A collection of papers on the application of Markov decision processes is surveyed and classified according to the use of real life data, structural results and special computational schemes. Observations are made about various features of the applications.

[1]  L. C. Thomas,et al.  A survey of maintenance and replacement models for maintainability and reliability of multi-item systems , 1986 .

[2]  A. Mandelbaum,et al.  Optimal Entering Rules for a Customer with Wait Option at an M/G/1 Queue , 1983 .

[3]  J Jaap Wessels,et al.  Dynamic planning of sales promotions by Markov programming , 1975 .

[4]  D. Rosenfield,et al.  Optimal Strategies for Selling an Asset , 1983 .

[5]  D. White.  Dynamic Programming and Systems of Uncertain Duration , 1965 .

[6]  Alejandro A. López-Toledo.  A controlled Markov chain model for nursing homes , 1976 .

[7]  J Jaap Wessels.  Markov Decision Processes. Implementation Aspects , 1981 .

[8]  Richard M. Feldman,et al.  Feature Article - Operations Research for Agricultural Pest Management , 1982, Oper. Res..

[9]  Matthew J. Sobel.  Making Short-Run Changes in Production when the Employment Level is Fixed , 1970, Oper. Res..

[10]  M. Rothstein.  An Airline Overbooking Model , 1971 .

[11]  Lakhbir S. Hayre.  A note on optimal maintenance policies for deciding whether to repair or replace , 1983 .

[12]  N. Hastings.  Optimal Claiming on Vehicle Insurance , 1976 .

[13]  S. Arunkumar,et al.  On Optimal Regulation Policies for Certain Multi-Reservoir Systems , 1978, Oper. Res..

[14]  D. J. White,et al.  Further Real Applications of Markov Decision Processes , 1988 .

[15]  Hirohide Hinomoto.  Sequential Control of Homogeneous Activities - Linear Programming of Semi-Markovian Decisions , 1971, Oper. Res..

[16]  James E. Eckles,et al.  Optimum Maintenance with Incomplete Information , 1968, Oper. Res..

[17]  Tom K. Lee.  On the Reswitching Property of R&D , 1982 .

[18]  André Turgeon.  A decomposition method for the long-term scheduling of reservoirs in series , 1981 .

[19]  A. Turgeon.  Optimal operation of multireservoir power systems with stochastic inflows , 1980 .

[20]  S. Yakowitz.  Dynamic programming applications in water resources , 1982 .

[21]  John M. Norman,et al.  Dynamic Programming in Tennis—When to Use a Fast Serve , 1985 .

[22]  L. Thomas,et al.  Optimal inspection policies for standby systems , 1987 .

[23]  Kamal Golabi,et al.  Optimal Inventory Policies When Ordering Prices are Random , 1985, Oper. Res..

[24]  John D. C. Little,et al.  The Use of Storage Water in a Hydroelectric System , 1955, Oper. Res..

[25]  Uri Yechiali,et al.  The hotel overbooking problem , 1983 .

[26]  James N. Eagle.  The Optimal Search for a Moving Target When the Search Path Is Constrained , 1984, Oper. Res..

[27]  H. Bierman,et al.  The Credit Granting Decision , 1970 .

[28]  R. Mendelssohn.  Discount Factors and Risk Aversion in Managing Random Fish Populations , 1982 .

[29]  B. F. Sule,et al.  Stochastic dynamic programming models for reservoir operation optimization , 1984 .

[30]  J. M. Norman,et al.  Control of Cash Reserves , 1965 .

[31]  Shaler Stidham,et al.  Control of arrivals to two queues in series , 1985 .

[32]  U. Shamir.  Application of operations research in Israel's water sector , 1980 .

[33]  Leon H. Liebman.  A Markov Decision Model for Selecting Optimal Credit Control Policies , 1972 .

[34]  André Turgeon,et al.  The Weekly Scheduling of Hydroelectric Power Plants Subject to Reliability Constraints , 1985 .

[35]  M. J. Sobel,et al.  Capital accumulation and the optimization of renewable resource models , 1980 .

[36]  Wayne L. Winston,et al.  Stochastic Control of Competition Through Prices , 1979, Oper. Res..

[37]  Stuart E. Dreyfus,et al.  A Note on an Industrial Replacement Process , 1957 .

[38]  G. Monahan.  Optimal Advertising with Stochastic Demand , 1983 .

[39]  Richard E. Rosenthal,et al.  Stochastic Dynamic Location Analysis , 1978 .

[40]  R. Deb.  Optimal Dispatching of a Finite Capacity Shuttle. , 1978 .

[41]  Edward P. C. Kao,et al.  Optimal Replacement Rules when Changes of State are Semi-Markovian , 1973, Oper. Res..

[42]  D. Onstad,et al.  Dynamic programming and the computation of economic injury levels for crop disease control , 1985 .

[43]  John Duncan,et al.  Interrupt and Opportunistic Replacement Strategies for Systems of Deteriorating Components , 1973 .

[44]  D. J. White,et al.  Real Applications of Markov Decision Processes , 1985 .

[45]  Dieter Bartmann.  Mittelfristige Produktionsplanung bei ungewissen Szenarien , 1984, Z. Oper. Research.

[46]  Christopher H. Nevison,et al.  Dynamic Lot-Sizing When Demand Timing Is Uncertain , 1984, Oper. Res..

[47]  Claude Lefèvre,et al.  Optimal Control of a Birth and Death Epidemic Process , 1981, Oper. Res..

[48]  M. Parlar.  Optimal Ordering Policies For A Perishable And Substitutable Product: A Markov Decision Model , 1985 .

[49]  Basil A. Kalymon.  Stochastic Prices in a Single-Item Inventory Purchasing Model , 1971, Oper. Res..

[50]  D. White.  Optimal revision periods , 1962 .

[51]  G. J. Lieberman,et al.  A Stochastic Sequential Allocation Model , 1975, Oper. Res..

[52]  Ralph D. Snyder.  Technical Note - A Dynamic Programming Formulation for Continuous Time Stock Control Systems , 1975, Oper. Res..

[53]  R. Mendelssohn.  Optimal harvesting strategies for stochastic single-species, multiage class models☆ , 1978 .

[54]  D. Bartmann.  The Optimal Regulation of the Cash-Balances of a Credit Institution , 1980 .

[55]  Michael Kolonko,et al.  The Sequential Design of Bernoulli Experiments Including Switching Costs , 1985, Oper. Res..

[56]  Yaron Ben-Ari,et al.  Optimal replacement policy for multicomponent systems: An application to a dairy herd , 1986 .

[57]  L. J. Thomas,et al.  Dynamic Models for Sales Promotion Policies , 1973 .

[58]  Roy Mendelssohn.  Managing stochastic multispecies models , 1980 .

[59]  Christopher Ellis,et al.  The application of dynamic programming in United Kingdom companies , 1974 .

[60]  Thomas B. Crabill,et al.  Optimal Control of a Maintenance System with Variable Service Rates , 1974, Oper. Res..

[61]  S Yakowitz,et al.  DYNAMIC PROGRAMMING APPLICATION IN WATER RESOURCES , 1982 .

[62]  David W. Low,et al.  Optimal Dynamic Pricing Policies for an M/M/s Queue , 1974, Oper. Res..

[63]  Roman Krzysztofowicz.  Optimal Water Supply Planning Based on Seasonal Runoff Forecasts , 1986 .

[64]  Peter J. Kolesar,et al.  Optimal Dispatching of an Infinite-Capacity Shuttle: Control at a Single Terminal , 1974, Oper. Res..

[65]  C. Bradley Russell,et al.  An Optimal Policy for Operating a Multipurpose Reservoir , 1972, Oper. Res..

[66]  David Kohler,et al.  Optimal Strategies for the Game of Darts , 1982 .

[67]  Marvin Rothstein.  HOTEL OVERBOOKING AS A MARKOVIAN SEQUENTIAL DECISION PROCESS , 1974 .

[68]  D. J. White.  An Example of Loosely Coupled Stages in Dynamic Programming , 1973 .

[69]  Uday S. Karmarkar.  The Multilocation Multiperiod Inventory Problem: Bounds and Approximations , 1987 .

[70]  S. Ladany.  DYNAMIC OPERATING RULES FOR MOTEL RESERVATIONS , 1976 .

[71]  N. Buras.  A Three-Dimensional Optimization Problem in Water-Resources Engineering , 1965 .

[72]  Milton C. Chew.  Optimal Stopping in a Discrete Search Problem , 1973, Oper. Res..

[73]  Vesa Saario.  Limiting properties of the discounted house-selling problem , 1985 .

[74]  J. Kolderman,et al.  Optimal Claiming in an Automobile Insurance System with Bonus-Malus Structure , 1985 .

[75]  P. Schweitzer,et al.  Part selection policy for a flexible manufacturing cell feeding several production lines , 1984 .

[76]  Minghui Yang,et al.  A hazardous-inspection model with costly repair , 1988 .

[77]  David L. Jaquette,et al.  Mathematical Models for Controlling Growing Biological Populations: A Survey , 1972, Oper. Res..

[78]  Hajime Kawai,et al.  An optimal inspection and replacement policy under incomplete state information , 1986 .

[79]  Gifford H. Symonds,et al.  Solution Method for a Class of Stochastic Scheduling Problems for the Production of a Single Commodity , 1971, Oper. Res..

[80]  Henk Tijms,et al.  A Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information , 1985 .

[81]  Lyn C. Thomas,et al.  The blast furnaces problem , 1980 .

[82]  S. Mann.  A mathematical theory for the harvest of natural animal populations when birth rates are dependent on total population size , 1970 .

[83]  B. Gluss.  An Optimum Policy for Detecting a Fault in a Complex System , 1959 .

[84]  Bhaskar Sengupta,et al.  Control limit policies for early detection of failure , 1981 .

[85]  Rolf A. Deininger,et al.  Generalization of White's Method of Successive Approximations to Periodic Markovian Decision Processes , 1972, Oper. Res..

[86]  S. Sarin,et al.  Determination of Optimal Pumping Policy of a Municipal Water Plant , 1982 .

[87]  Sheldon M. Ross,et al.  A Problem in Optimal Search and Stop , 1969, Oper. Res..