Dynamic Inventory Repositioning in On-Demand Rental Networks

We consider a rental service with a fixed number of rental units distributed across multiple locations. The units are accessed by customers without prior reservation and on an on-demand basis. Customers can decide on how long to keep a unit and where to return it. Because of the randomness in demand and in returns, there is a need to periodically reposition inventory away from some locations and into others. In deciding on how much inventory to reposition and where, the system manager balances potential lost sales with repositioning costs. Although the problem is increasingly common in applications involving on-demand rental services, not much is known about the nature of the optimal policy for systems with a general network structure or about effective approaches to solving the problem. In this paper, first, we show that the optimal policy in each period can be described in terms of a well-specified region over the state space. Within this region, it is optimal not to reposition any inventory, while, outside the region, it is optimal to reposition but only such that the system moves to a new state that is on the boundary of the no-repositioning region. We also provide a simple check for when a state is in the no-repositioning region. Second, we leverage the features of the optimal policy, along with properties of the optimal cost function, to propose a provably convergent approximate dynamic programming algorithm to tackle problems with a large number of dimensions.

[1]  Maurizio Bruglieri,et al.  The vehicle relocation problem for the one-way electric vehicle sharing , 2013, ArXiv.

[2]  Csaba Szepesvári,et al.  Finite-Time Bounds for Fitted Value Iteration , 2008, J. Mach. Learn. Res..

[3]  Pierre Girardeau,et al.  On the Convergence of Decomposition Methods for Multistage Stochastic Convex Programs , 2015, Math. Oper. Res..

[4]  Nan Jiang,et al.  The Dependence of Effective Planning Horizon on Model Accuracy , 2015, AAMAS.

[5]  Long He,et al.  Service Region Design for Urban Electric Vehicle Sharing Systems , 2017, Manuf. Serv. Oper. Manag..

[6]  Ying Rong,et al.  Operations Management of Vehicle Sharing Systems , 2019 .

[7]  Robert C. Hampshire,et al.  Inventory rebalancing and vehicle routing in bike sharing systems , 2017, Eur. J. Oper. Res..

[8]  Long He,et al.  Robust Repositioning for Vehicle Sharing , 2020, Manuf. Serv. Oper. Manag..

[9]  Frances Sprei,et al.  Comparison of free-floating car-sharing services in cities , 2017 .

[10]  Hui Xiong,et al.  Rebalancing Bike Sharing Systems: A Multi-source Data Smart Optimization , 2016, KDD.

[11]  John R. Birge,et al.  Successive Linear Approximation Solution of Infinite-Horizon Dynamic Stochastic Programs , 2007, SIAM J. Optim..

[12]  Thodoris Lykouris,et al.  Pricing and Optimization in Shared Vehicle Systems: Queueing models and Approximation Algorithms , 2016 .

[13]  Warren B. Powell,et al.  Dynamic Programming Models and Algorithms for the Mutual Fund Cash Balance Problem , 2010, Manag. Sci..

[14]  Alice Paul,et al.  Data-Driven Rebalancing Methods for Bike-Share Systems , 2020 .

[15]  D. Shmoys,et al.  Bike Sharing , 2019, Springer Series in Supply Chain Management.

[16]  Warrren B Powell,et al.  Convergent Cutting-Plane and Partial-Sampling Algorithm for Multistage Stochastic Linear Programs with Recourse , 1999 .

[17]  Patrick Jaillet,et al.  Dynamic Repositioning to Reduce Lost Demand in Bike Sharing Systems , 2017, J. Artif. Intell. Res..

[18]  Evan L. Porteus Conditions for characterizing the structure of optimal strategies in infinite-horizon dynamic programs , 1982 .

[19]  Karan Girotra,et al.  Bike-Share Systems: Accessibility and Availability , 2018, Manag. Sci..

[20]  Alexander Shapiro,et al.  Analysis of stochastic dual dynamic programming method , 2011, Eur. J. Oper. Res..

[21]  Sunil Kumar,et al.  MULTIDIMENSIONAL PORTFOLIO OPTIMIZATION WITH PROPORTIONAL TRANSACTION COSTS , 2006 .

[22]  Shane Legg,et al.  Human-level control through deep reinforcement learning , 2015, Nature.

[23]  Elise Miller-Hooks,et al.  Fleet Management for Vehicle Sharing Operations , 2011, Transp. Sci..

[24]  U. Karmarkar,et al.  The one‐period, N‐location distribution problem , 1977 .

[25]  David B. Shmoys,et al.  Minimizing Multimodular Functions and Allocating Capacity in Bike‐Sharing Systems , 2016, IPCO.

[26]  Evan L. Porteus On the Optimality of Structured Policies in Countable Stage Decision Processes , 1975 .

[27]  Warrren B Powell,et al.  An Adaptive, Distribution-Free Algorithm for the Newsvendor Problem with Censored Demands, with Applications to Inventory and Distribution , 2001 .

[28]  W. Y. Szeto,et al.  Dynamic green bike repositioning problem – A hybrid rolling horizon artificial bee colony algorithm approach , 2017 .

[29]  Warren B. Powell,et al.  Learning Algorithms for Separable Approximations of Discrete Stochastic Optimization Problems , 2004, Math. Oper. Res..

[30]  Dong-Ping Song Optimal threshold control of empty vehicle redistribution in two depot service systems , 2005, IEEE Transactions on Automatic Control.

[31]  Warren B. Powell,et al.  An Optimal Approximate Dynamic Programming Algorithm for the Lagged Asset Acquisition Problem , 2009, Math. Oper. Res..

[32]  Christine Fricker,et al.  Incentives and redistribution in homogeneous bike-sharing systems with stations of finite capacity , 2012, EURO J. Transp. Logist..

[33]  Vincent Jost,et al.  Pricing in vehicle sharing systems: optimization in queuing networks with product forms , 2016, EURO J. Transp. Logist..

[34]  Zhihao Chen,et al.  Optimizing the Profitability and Quality of Service in Carshare Systems Under Demand Uncertainty , 2018, Manuf. Serv. Oper. Manag..

[35]  David B. Shmoys,et al.  Data Analysis and Optimization for (Citi)Bike Sharing , 2015, AAAI.

[36]  David B. Shmoys,et al.  Bike Angels: An Analysis of Citi Bike's Incentive Program , 2018, COMPASS.

[37]  Panos M. Pardalos,et al.  Approximate dynamic programming: solving the curses of dimensionality , 2009, Optim. Methods Softw..

[38]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[39]  Enver Yücesan,et al.  The Multi-Location Transshipment Problem , 2004 .

[40]  Ling Zhao,et al.  Dynamic repositioning for vehicle sharing with setup costs , 2020, Oper. Res. Lett..

[41]  James E. Smith,et al.  Dynamic Portfolio Optimization with Transaction Costs: Heuristics and Dual Bounds , 2011, Manag. Sci..

[42]  Zhaolin Li,et al.  On determining optimal fleet size and vehicle transfer policy for a car rental company , 2010, Comput. Oper. Res..

[43]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[44]  Oded Berman,et al.  Optimal Joint Replenishment and Transshipment Policies in a Multi-Period Inventory System With Lost Sales , 2015 .

[45]  Andrew B. Philpott,et al.  On the convergence of stochastic dual dynamic programming and related methods , 2008, Oper. Res. Lett..

[46]  Susan R. Hunter,et al.  A Bound on the Performance of an Optimal Ambulance Redeployment Policy , 2014, Oper. Res..

[47]  Andy Philpott,et al.  On the Convergence of Sampling-Based Decomposition Algorithms for Multistage Stochastic Programs , 2005 .

[48]  Ke Liu,et al.  Allocation of empty containers between multi-ports , 2007, Eur. J. Oper. Res..

[49]  Thodoris Lykouris,et al.  Pricing and Optimization in Shared Vehicle Systems: An Approximation Framework , 2016, EC.

[50]  Lawrence W. Robinson,et al.  Optimal and Approximate Policies in Multiperiod, Multilocation Inventory Models with Transshipments , 1990, Oper. Res..

[51]  Ruud H. Teunter,et al.  Inventory models with lateral transshipments: A review , 2011, Eur. J. Oper. Res..

[52]  Yu Zheng,et al.  Dynamic Bike Reposition: A Spatio-Temporal Reinforcement Learning Approach , 2018, KDD.

[53]  Jan A. Van Mieghem,et al.  Commissioned Paper: Capacity Management, Investment, and Hedging: Review and Recent Developments , 2003, Manuf. Serv. Oper. Manag..

[54]  I-Lin Wang,et al.  Models for Effective Deployment and Redistribution of Bicycles Within Public Bicycle-Sharing Systems , 2013, Oper. Res..

[55]  Benjamin Van Roy,et al.  On the existence of fixed points for approximate value iteration and temporal-difference learning , 2000 .

[56]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[57]  John Lygeros,et al.  Generalized Dual Dynamic Programming for Infinite Horizon Problems in Continuous State and Action Spaces , 2017, IEEE Transactions on Automatic Control.

[58]  Siddhartha Banerjee,et al.  Dynamic Assignment Control of a Closed Queueing Network under Complete Resource Pooling , 2018 .

[59]  Cathy H. Xia,et al.  Fleet-sizing and service availability for a vehicle rental system via closed queueing networks , 2011, Eur. J. Oper. Res..

[60]  Sean X. Zhou,et al.  Joint Initial Stocking and Transshipment—Asymptotics and Bounds , 2016 .

[61]  M. V. F. Pereira,et al.  Multi-stage stochastic optimization applied to energy planning , 1991, Math. Program..

[62]  Izak Duenyas,et al.  Optimal Joint Inventory and Transshipment Control Under Uncertain Capacity , 2005, Oper. Res..

[63]  Warren B. Powell,et al.  An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, I: Single Period Travel Times , 2002, Transp. Sci..

[64]  Matthew S. Maxwell,et al.  Approximate Dynamic Programming for Ambulance Redeployment , 2010, INFORMS J. Comput..

[65]  Tal Raviv,et al.  Bike-sharing systems: User dissatisfaction in the presence of unusable bicycles , 2017 .

[66]  David C. Parkes,et al.  Spatio-Temporal Pricing for Ridesharing Platforms , 2018, EC.

[67]  Long Ji Lin,et al.  Self-improving reactive agents based on reinforcement learning, planning and teaching , 1992, Machine Learning.

[68]  Jacob B. Feldman,et al.  Free Rides in Dockless, Electric Vehicle Sharing Systems , 2019, SSRN Electronic Journal.

[69]  Warren B. Powell,et al.  Dynamic Control of Logistics Queueing Networks for Large-Scale Fleet Management , 1998, Transp. Sci..

[70]  Uday S. Karmarkar,et al.  The Multiperiod Multilocation Inventory Problem , 1981, Oper. Res..

[71]  Julia L. Higle,et al.  Stochastic Decomposition: An Algorithm for Two-Stage Linear Programs with Recourse , 1991, Math. Oper. Res..

[72]  Shane G. Henderson,et al.  Simulation optimization for a large-scale bike-sharing system , 2016, 2016 Winter Simulation Conference (WSC).

[73]  Qiang Meng,et al.  Handbook of Ocean Container Transport Logistics , 2015 .

[74]  Chen Chen,et al.  Dynamic Pricing of Relocating Resources in Large Networks , 2019, Manag. Sci..

[75]  Tal Raviv,et al.  Optimal inventory management of a bike-sharing station , 2013 .

[76]  Saif Benjaafar,et al.  Dimensioning On-Demand Vehicle Sharing Systems , 2020, Manag. Sci..

[77]  Kostas Bimpikis,et al.  Spatial pricing in ride-sharing networks , 2016, NetEcon@EC.