Adaptive Inventory Control for Nonstationary Demand and Partial Information

This paper examines several different policies for an inventory control problem in which the demand process is nonstationary and partially observed. The probability distribution for the demand in each period is determined by the state of a Markov chain, the core process. However, the state of this core process is not directly observed, only the actual demand is observed by the decision maker. Given this demand process, the inventory control problem is a composite-state, partially observed Markov decision process POMDP, which is an appropriate model for a number of dynamic demand problems. In practice, managers often use certainty equivalent control CEC policies to solve such a problem. However, this paper presents results that demonstrate that there are other practical control policies that almost always provide much better solutions for this problem than the CEC policies commonly used in practice. The computational results also indicate how specific problem characteristics influence the performance of each of the alternative policies.

[1]  M. Puterman,et al.  Maximum-penalized-likelihood estimation for independent and Markov-dependent mixture models. , 1992, Biometrics.

[2]  H. Scarf.  Some remarks on bayes solutions to the inventory problem , 1960 .

[3]  P. Schweitzer.  Contraction mappings underlying undiscounted Markov decision problems—II , 1978 .

[4]  K. Arrow,et al.  Optimal Inventory Policy. , 1951 .

[5]  Katy S. Azoury.  Bayes Solution to Dynamic Inventory Models Under Unknown Demand Distribution , 1985 .

[6]  Chelsea C. White,et al.  Solution Procedures for Partially Observed Markov Decision Processes , 1989, Oper. Res..

[7]  T. Morton,et al.  Discounting, Ergodicity and Convergence for Markov Decision Processes , 1977 .

[8]  T. M. Whitin,et al.  An Optimal Final Inventory Model , 1961 .

[9]  Robert L. Smith,et al.  Conditions for the Existence of Planning Horizons , 1984, Math. Oper. Res..

[10]  H. Scarf.  Bayes Solutions of the Statistical Inventory Problem , 1959 .

[11]  Thomas E. Morton,et al.  The Nonstationary Stochastic Lead-Time Inventory Problem: Near-Myopic Bounds, Heuristics, and Testing , 1996 .

[12]  S. Karlin.  Dynamic Inventory Policy with Varying Stochastic Demands , 1960 .

[13]  William S. Lovejoy,et al.  Some Monotonicity Results for Partially Observed Markov Decision Processes , 1987, Oper. Res..

[14]  G. Monahan.  State of the Art—A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms , 1982 .

[15]  David K. Smith,et al.  Dynamic Programming and Optimal Control. Volume 1 , 1996 .

[16]  Jing-Sheng Song,et al.  Inventory Control in a Fluctuating Demand Environment , 1993, Oper. Res..

[17]  William S. Lovejoy,et al.  Suboptimal Policies, with Bounds, for Parameter Adaptive Decision Processes , 1993, Oper. Res..

[18]  D. Rhenius.  Incomplete Information in Markovian Decision Models , 1974 .

[19]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[20]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[21]  Stephen A. Smith,et al.  Estimating negative binomial demand for retail inventory management with unobservable lost sales , 1996 .

[22]  W. Lovejoy.  Stopped Myopic Policies in Some Inventory Models with Generalized Demand Processes , 1992 .

[23]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[24]  Hirofumi Matsuo,et al.  Forecasting and Inventory Management of Short Life-Cycle Products , 1996, Oper. Res..

[25]  Stephen C. Graves,et al.  A Single-Item Inventory Model for a Nonstationary Demand Process , 1999, Manuf. Serv. Oper. Manag..

[26]  W. Lovejoy.  A survey of algorithmic methods for partially observed Markov decision processes , 1991 .

[27]  Chelsea C. White,et al.  Finite-Memory Suboptimal Design for Partially Observed Markov Decision Processes , 1994, Oper. Res..

[28]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[29]  Edward J. Sondik,et al.  The Optimal Control of Partially Observable Markov Processes over a Finite Horizon , 1973, Oper. Res..

[30]  Steven L. Shafer,et al.  Comparison of Some Suboptimal Control Policies in Medical Drug Therapy , 1996, Oper. Res..

[31]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[32]  W. Lovejoy.  Myopic policies for some inventory models with uncertain demand distributions , 1990 .

[33]  Jing-Sheng Song,et al.  Managing Inventory with the Prospect of Obsolescence , 1996, Oper. Res..