Inventory management with partially observed nonstationary demand

We consider a continuous-time model for inventory management with Markov modulated non-stationary demands. We introduce active learning by assuming that the state of the world is unobserved and must be inferred by the manager. We also assume that demands are observed only when they are completely met. We first derive the explicit filtering equations and pass to an equivalent fully observed impulse control problem in terms of the sufficient statistics, the a posteriori probability process and the current inventory level. We then solve this equivalent formulation and directly characterize an optimal inventory policy. We also describe a computational procedure to calculate the value function and the optimal policy and present two numerical illustrations.

[1]  A. Bensoussan,et al.  On the optimal control of partially observed inventory systems , 2005 .

[2]  Evan L. Porteus,et al.  Stalking Information: Bayesian Inventory Management with Unobserved Lost Sales , 1999 .

[3]  Pierre Bertrand,et al.  Discrete-time estimation of a Markov chain with marked point process observations. Application to Markovian jump filtering , 2001, IEEE Trans. Autom. Control..

[4]  Erhan Bayraktar,et al.  Sequential tracking of a hidden Markov chain using point process observations , 2007, 0712.0413.

[5]  S. Sethi,et al.  Average Cost Optimality in Inventory Models with Markovian Demands and Lost Sales , 2003 .

[6]  J. Darroch,et al.  Passage-time generating functions for continuous-time finite Markov chains , 1968, Journal of Applied Probability.

[7]  P. Brémaud.  Point Processes and Queues , 1981 .

[8]  Suresh P. Sethi,et al.  Optimality of (s, S) Policies in Inventory Models with Markovian Demand , 1995 .

[9]  S. Karlin,et al.  A second course in stochastic processes , 1981 .

[10]  Jing-Sheng Song,et al.  Inventory Control in a Fluctuating Demand Environment , 1993, Oper. Res..

[11]  Katy S. Azoury.  Bayes Solution to Dynamic Inventory Models Under Unknown Demand Distribution , 1985 .

[12]  Suresh P. Sethi,et al.  Partially Observed Inventory Systems: The Case of Zero-Balance Walk , 2007, SIAM J. Control. Optim..

[13]  Suresh P. Sethi,et al.  Optimality of an (s, S) Policy with Compound Poisson and Diffusion Demands: A Quasi-Variational Inequalities Approach , 2005, SIAM J. Control. Optim..

[14]  Mark H. Davis.  Markov Models and Optimization , 1995 .

[15]  Robert J. Elliott,et al.  Robust M-ary detection filters and smoothers for continuous-time jump Markov systems , 2004, IEEE Transactions on Automatic Control.

[16]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[17]  Dariusz Gatarek,et al.  Optimality conditions for impulsive control of piecewise-deterministic processes , 1992, Math. Control. Signals Syst..

[18]  S. P. Sethi,et al.  Inventory Problems with Partially Observed Demands and Lost Sales , 2008 .

[19]  Robert J. Elliott,et al.  General smoothing formulas for Markov-modulated Poisson observations , 2005, IEEE Transactions on Automatic Control.

[20]  O. Hernández-Lerma.  Adaptive Markov Control Processes , 1989 .

[21]  Elja Arjas,et al.  Filtering the histories of a partially observed marked point process , 1992 .

[22]  Avner Friedman,et al.  Optimal Stopping Problems in Stochastic Control , 1979 .

[23]  B. Conolly.  Structured Stochastic Matrices of M/G/1 Type and Their Applications , 1991 .

[24]  C. R. Sox,et al.  Adaptive Inventory Control for Nonstationary Demand and Partial Information , 2002, Manag. Sci..

[25]  Yossi Aviv,et al.  A Partially Observed Markov Decision Process for Dynamic Pricing , 2005, Manag. Sci..

[26]  M. Ludkovski,et al.  Finite Horizon Decision Timing with Partially Observable Poisson Processes , 2011, 1105.1484.

[27]  Erhan Bayraktar,et al.  Quickest Detection for a Poisson Process with a Phase-type Change-time Distribution , 2006 .

[28]  Alain Bensoussan,et al.  On the Optimal Control of Partially Observed Inventory Systems , 2005 .

[29]  Marcel F. Neuts,et al.  Structured Stochastic Matrices of M/G/1 Type and Their Applications , 1989 .

[30]  M. H. A. Davis,et al.  Impulse control of piecewise-deterministic processes , 1989, Math. Control. Signals Syst..

[31]  Suresh P. Sethi,et al.  Optimality of (s, S) Policies in Inventory Models with Markovian Demand , 1995, Oper. Res..

[32]  Alain Bensoussan,et al.  Optimality of an (s, S) Policy with Compound Poisson and Diffusion Demands: A Quasi-Variational Inequalities Approach , 2009, SIAM J. Control. Optim..

[33]  S. Eddy.  Hidden Markov models. , 1996, Current opinion in structural biology.

[34]  W. Lovejoy.  Myopic policies for some inventory models with uncertain demand distributions , 1990 .

[35]  J. Craggs.  Applied Mathematical Sciences , 1973 .

[36]  Suresh P. Sethi,et al.  A Multiperiod Newsvendor Problem with Partially Observed Demand , 2007, Math. Oper. Res..