A Metastrategy for Large-Scale Resource Management Based on Informational Decomposition

This paper addresses the solution of large, complex resource allocation problems, examples of which include large freight transportation companies and supply chain management. Some instances of these problems involve millions of constraints and tens of millions of variables. Classical formulations focus on modeling the physical problem alone. In this paper, we focus on modeling the organization of information and decisions, producing a natural decomposition based on how decisions are actually made. Restricting the size of a subproblem to the sizes of problems actually solved by real decision makers, we avoid the computational demands posed by large problems. The algorithmic challenge is producing high quality solutions that reflect the interaction between subproblems. Linear approximations have been a widely used tool for decomposition, but these can produce unstable solutions of only moderate quality. We introduce the concept of using nonlinear approximations, which creates special technical problems but also produces solutions of very high quality. The strategy is simulated on two problem classes (fleet management and supply chains) and compared against standard modeling strategies. Synchronous and asynchronous strategies are also compared.

[1]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[2]  Warren B. Powell,et al.  An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, I: Single Period Travel Times , 2002, Transp. Sci..

[3]  Warrren B Powell,et al.  An adaptive dynamic programming algorithm for a stochastic multiproduct batch dispatch problem , 2003 .

[4]  B PowellWarren,et al.  An Adaptive Dynamic Programming Algorithm for Dynamic Fleet Management, II , 2002 .

[5]  Warren B. Powell,et al.  Stochastic programs over trees with random arc capacities , 1994, Networks.

[6]  F. J. Gould,et al.  Extensions of the Planning Horizon Theorem in the Dynamic Lot Size Model , 1969 .

[7]  Yuri Ermoliev,et al.  Numerical techniques for stochastic optimization , 1988 .

[8]  Warren B. Powell,et al.  A framework for representing and solving dynamic resource transformation problems , 1999 .

[9]  Linos F. Frantzeskakis,et al.  A Successive Linear Approximation Procedure for Stochastic, Dynamic Vehicle Allocation Problems , 1990, Transp. Sci..

[10]  Warren B. Powell,et al.  Dynamic Control of Logistics Queueing Networks for Large-Scale Fleet Management , 1998, Transp. Sci..

[11]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[12]  Warren B. Powell,et al.  An Algorithm for Multistage Dynamic Networks with Random Arc Capacities, with an Application to Dynamic Fleet Management , 1996, Oper. Res..

[13]  Teodor Gabriel Crainic,et al.  Survey Paper - A Review of Empty Flows and Fleet Management Models in Freight Transportation , 1987, Transp. Sci..

[14]  Warren B. Powell,et al.  Exploiting structure in adaptive dynamic programming algorithms for a stochastic batch service problem , 2002, Eur. J. Oper. Res..

[15]  Stephen C. Graves Multistage Lot-Sizing: An Iterative Procedure. , 1979 .

[16]  Warren B. Powell,et al.  A Representational Paradigm for Dynamic Resource Transformation Problems , 2001, Ann. Oper. Res..

[17]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[18]  Warrren B Powell,et al.  An Adaptive, Distribution-Free Algorithm for the Newsvendor Problem with Censored Demands, with Applications to Inventory and Distribution , 2001 .

[19]  Warren B. Powell,et al.  An Approximation Algorithm for Solving Multistage, Stochastic, Dynamic Resource Allocation Problems with Single Period Travel Times , 1999 .

[20]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[21]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[22]  W. Zangwill A Backlogging Model and a Multi-Echelon Model of a Dynamic Economic Lot Size Production System---A Network Approach , 1969 .

[23]  Harvey M. Wagner,et al.  Dynamic Version of the Economic Lot Size Model , 2004, Manag. Sci..