Case-based myopic reinforcement learning for satisfying target service level in supply chain

In the last decade, driven by global competition in the marketplace, many companies have taken initiatives to revamp their supply chains in order to increase responsiveness to changes in the marketplace. The renovation of inventory control system is central to such an effort. However, experiences in industry have shown that the control of inventory in supply chain is not an easy task because of uncertainties inherent in customer demand. In this paper, we propose a reinforcement learning algorithm appropriate for the nonstationary inventory control problem of supply chain that has a large state space. Traditional reinforcement learning algorithms such as learning automata and Q-learning have the difficulty of slow convergence when applied to the situations with large state spaces. To resolve the problems of nonstationary customer demand and large state space, we develop a case-based myopic reinforcement learning (CMRL) algorithm. A simulation-based experiment was performed to show good performance of CMRL.

[1]  Riikka Kaipia,et al.  VMI: What are you losing if you let your customer place orders? , 2002 .

[2]  Jan Holmström,et al.  Implementing vendor-managed inventory the efficient way: a case study of partnership in the supply chain , 1998 .

[3]  de Ag Ton Kok,et al.  Distribution planning for a divergent depotless two-echelon network under service constraints , 1996 .

[4]  Roger C. Vergin,et al.  Building Competitiveness in Grocery Supply Through Continuous Replenishment Planning , 1999 .

[5]  Kumpati S. Narendra,et al.  Learning automata - an introduction , 1989 .

[6]  Eb Erik Diks,et al.  Multi-echelon systems: A service measure perspective , 1996 .

[7]  Graham K. Rand,et al.  Logistics of Production and Inventory , 1995 .

[8]  Ashwin Ram,et al.  Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces , 1997, Adapt. Behav..

[9]  Paul H. Zipkin,et al.  Foundations of Inventory Management , 2000 .

[10]  Robert Goodell Brown,et al.  Smoothing, forecasting and prediction of discrete time series , 1964 .

[11]  Stephen A. Smith,et al.  A decision support system for vendor managed inventory , 2000 .

[12]  S. Chopra,et al.  Supply Chain Management: Strategy, Planning & Operation , 2007 .

[13]  Chung-Yee Lee,et al.  Stock Replenishment and Shipment Scheduling for Vendor-Managed Inventory Systems , 2000 .

[14]  R. J. Tersine Principles of inventory and materials management , 1982 .

[15]  Yeong-Dae Kim,et al.  Adaptive inventory control models for supply chain management , 2005 .

[16]  Abhijit Gosavi,et al.  Global supply chain management: A reinforcement learning approach , 2002 .

[17]  Yan Dong,et al.  A supply chain model of vendor managed inventory , 2002 .

[18]  Jinwoo Park,et al.  Vendor Managed Inventory and Its Effect in the Supply Chain , 2004, AsiaSim.

[19]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .