Decentralized control of a stochastic dynamic resource allocation problem

This paper concerns decentralized control of a certain stochastic and dynamic resource allocation problem. The system consists of two agents, a pricing agent and a service agent. The pricing agent controls the customer arrival rate by dynamically setting prices while the service agent controls the rate at which customers are served. The distinguishing feature of our setup is that the pricing and service agents know different subcomponents of the model but are unwilling or unable to reveal their knowledge of the system to the other agent (or a centralized controller). This means that joint optimization over pricing and service policies is not possible since there is no single agent with knowledge of all relevant system parameters. Within this setup, we show that the centralized optimal pricing and service policies can still be constructed. Specifically, the integrated problem can be decoupled into a dynamic pricing problem (for the pricing agent) and a service rate control problem (for the service agent), and that these single-agent problems can be specified so as to deliver the centralized optimal policies. We also present an iterative algorithm which enables pricing and service agents to construct the centralized optimal policies without having to reveal private knowledge about the system to the other agent.

[1]  M. Athans,et al.  Solution of some nonclassical LQG stochastic decision problems , 1974 .

[2]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[3]  Dimitri P. Bertsekas,et al.  Separable Dynamic Programming and Approximate Decomposition Methods , 2007, IEEE Transactions on Automatic Control.

[4]  G. Ryzin,et al.  Optimal dynamic pricing of inventories with stochastic demand over finite horizons , 1994 .

[5]  B. Kurtaran.  Dynamic two-person two-objective control problems with delayed sharing information pattern , 1977 .

[6]  R. Radner,et al.  Team Decision Problems , 1962 .

[7]  P. Varaiya.  Optimal and suboptimal stationary controls for Markov chains , 1978 .

[8]  David G. Luenberger,et al.  Linear and Nonlinear Programming: Second Edition , 2003 .

[9]  J. Marschak,et al.  Elements for a Theory of Teams , 1955 .

[10]  J. Walrand,et al.  On delayed sharing patterns , 1978 .

[11]  H. Witsenhausen.  A Counterexample in Stochastic Optimum Control , 1968 .

[12]  P. Brémaud.  Point Processes and Queues , 1981 .

[13]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[14]  Sanjay Lall,et al.  A Characterization of Convex Problems in Decentralized Control$^ast$ , 2005, IEEE Transactions on Automatic Control.

[15]  Daniel Adelman,et al.  Relaxations of Weakly Coupled Stochastic Dynamic Programs , 2008, Oper. Res..

[16]  Michael Athans,et al.  Survey of decentralized control methods for large scale systems , 1978 .

[17]  P. Brémaud.  Point processes and queues, martingale dynamics , 1983 .

[18]  Anders Rantzer,et al.  Dynamic dual decomposition for distributed control , 2009, 2009 American Control Conference.

[19]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[20]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[21]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[22]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[23]  Yu-Chi Ho.  Review of the Witsenhausen problem , 2008, 2008 47th IEEE Conference on Decision and Control.

[24]  Andrew E. B. Lim,et al.  Robust Multi-Product Pricing , 2008 .

[25]  Yu-Chi Ho,et al.  Team decision theory and information structures , 1980 .