Using Stochastic Approximation Methods to Compute Optimal Base-Stock Levels in Inventory Control Problems

In this paper, we consider numerous inventory control problems for which the base-stock policies are known to be optimal, and we propose stochastic approximation methods to compute the optimal base-stock levels. The existing stochastic approximation methods in the literature guarantee that their iterates converge, but not necessarily to the optimal base-stock levels. In contrast, we prove that the iterates of our methods converge to the optimal base-stock levels. Moreover, our methods continue to enjoy the well-known advantages of the existing stochastic approximation methods. In particular, they only require the ability to obtain samples of the demand random variables, rather than to compute expectations explicitly, and they are applicable even when the demand information is censored by the amount of available inventory.

[1]  Paul Glasserman.  Perturbation Analysis of Production Networks , 1994 .

[2]  Katy S. Azoury.  Bayes Solution to Dynamic Inventory Models Under Unknown Demand Distribution , 1985 .

[3]  S. Fotopoulos.  Stochastic modeling and analysis of manufacturing systems , 1996 .

[4]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[5]  David J. Braden,et al.  Informational dynamics of censored observations , 1991 .

[6]  J. Neveu,et al.  Discrete Parameter Martingales , 1975 .

[7]  M. Fu,et al.  Optimization of( s, S ) inventory systems with random lead times and a service level constraint , 1998 .

[8]  Garrett J. van Ryzin,et al.  Simulation-Based Optimization of Virtual Nesting Controls for Network Revenue Management , 2008, Oper. Res..

[9]  D. Iglehart.  The Dynamic Inventory Problem with Unknown Demand Distribution , 1964 .

[10]  Harold J. Kushner,et al.  wchastic. approximation methods for constrained and unconstrained systems , 1978 .

[11]  S. Karlin,et al.  Studies in the Mathematical Theory of Inventory and Production, by K.J. Arrow, S. Karlin, H. Scarf with contributions by M.J. Beckmann, J. Gessford, R.F. Muth. Stanford, California, Stanford University Press, 1958, X p.340p., $ 8.75. , 1959, Bulletin de l'Institut de recherches économiques et sociales.

[12]  Itir Z. Karaesmen,et al.  Overbooking with Substitutable Inventory Classes , 2004, Oper. Res..

[13]  P. Glynn,et al.  Stochastic Optimization by Simulation: Convergence Proofs for the GI/G/1 Queue in Steady-State , 1994 .

[14]  P. Glynn,et al.  Stochastic optimization by simulation: numerical experiments with the M / M /1 queue in steady-state , 1994 .

[15]  Warren B. Powell,et al.  An algorithm for approximating piecewise linear concave functions from sample gradients , 2003, Oper. Res. Lett..

[16]  Paul H. Zipkin,et al.  Foundations of Inventory Management , 2000 .

[17]  Garrett J. van Ryzin,et al.  Stocking Retail Assortments Under Dynamic Consumer Substitution , 2001, Oper. Res..

[18]  Michael C. Fu,et al.  Sample Path Derivatives for (s, S) Inventory Systems , 1994, Oper. Res..

[19]  Paul Glasserman,et al.  Sensitivity Analysis for Base-Stock Levels in Multiechelon Production-Inventory Systems , 1995 .

[20]  S. Karlin,et al.  Studies in the Mathematical Theory of Inventory and Production, by K.J. Arrow, S. Karlin, H. Scarf with contributions by M.J. Beckmann, J. Gessford, R.F. Muth. Stanford, California, Stanford University Press, 1958, X p.340p., $ 8.75. , 1959, Bulletin de l'Institut de recherches économiques et sociales.

[21]  Warren B. Powell,et al.  An Optimal Approximate Dynamic Programming Algorithm for the Lagged Asset Acquisition Problem , 2009, Math. Oper. Res..

[22]  S. A. Conrad,et al.  Sales Data and the Estimation of Demand , 1976 .

[23]  Warrren B Powell,et al.  An Adaptive, Distribution-Free Algorithm for the Newsvendor Problem with Censored Demands, with Applications to Inventory and Distribution , 2001 .

[24]  H. Scarf.  Some remarks on bayes solutions to the inventory problem , 1960 .

[25]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[26]  George L. Nemhauser,et al.  Handbooks in operations research and management science , 1989 .

[27]  John N. Tsitsiklis,et al.  Asynchronous Stochastic Approximation and Q-Learning , 1994, Machine Learning.

[28]  Warren B. Powell,et al.  Learning Algorithms for Separable Approximations of Discrete Stochastic Optimization Problems , 2004, Math. Oper. Res..