Decentralized Control of a Stochastic Multi-Agent Queueing System

This paper concerns decentralized control of a multi-agent stochastic queueing system. The system consists of two agents, a pricing agent and a service agent. The pricing agent controls the arrival rate by dynamically setting the fee that is paid by customers who would like to receive service, while the service agent controls the rate at which customers in the system are processed. Each agent optimizes his/her own objective subject to a model which accurately specifies the component of the system that he/she is controlling but potentially mis-specifies the component that is controlled by the other agent. We are interested in optimizing the efficiency of the aggregate system when control decisions are made in this way. Decentralized agents are coordinated using transfer contracts, which define a price for occupying unit of buffer space in the system as a function of time and available capacity. Transfer contracts lead to cash transfers between agents whenever a customer enters or departs, and serve to modify each of their incentives. We characterize the contract that maximizes the efficiency of the aggregate system under decentralized control, and show that it achieves the efficiency of a centralized agent who jointly optimizes over pricing and service policies with an accurate model of the integrated system. We also show that optimal centralized efficiency is robust to mis-specification by each agent of the dynamics induced by the other in his/her model. An iterative decentralized algorithm for constructing the optimal contract and a proof of convergence is also presented.

[1]  M. Athans,et al.  Solution of some nonclassical LQG stochastic decision problems , 1974 .

[2]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[3]  David G. Luenberger,et al.  Linear and Nonlinear Programming: Second Edition , 2003 .

[4]  R. Radner,et al.  Team Decision Problems , 1962 .

[5]  J. Marschak,et al.  Elements for a Theory of Teams , 1955 .

[6]  B. V. Dean,et al.  Studies in Linear and Non-Linear Programming. , 1959 .

[7]  Andrew E. B. Lim,et al.  Decentralized control of a multi-agent stochastic dynamic resource allocation problem , 2011, IEEE Conference on Decision and Control and European Control Conference.

[8]  P. Brémaud.  Point processes and queues, martingale dynamics , 1983 .

[9]  Anders Rantzer,et al.  Dynamic dual decomposition for distributed control , 2009, 2009 American Control Conference.

[10]  Steven H. Low,et al.  Optimization flow control—I: basic algorithm and convergence , 1999, TNET.

[11]  Michael Athans,et al.  Survey of decentralized control methods for large scale systems , 1978 .

[12]  Dimitri P. Bertsekas,et al.  Separable Dynamic Programming and Approximate Decomposition Methods , 2007, IEEE Transactions on Automatic Control.

[13]  J. Walrand,et al.  On delayed sharing patterns , 1978 .

[14]  H. Witsenhausen.  A Counterexample in Stochastic Optimum Control , 1968 .

[15]  B. Kurtaran.  Dynamic two-person two-objective control problems with delayed sharing information pattern , 1977 .

[16]  Sanjay Lall,et al.  A Characterization of Convex Problems in Decentralized Control$^ast$ , 2005, IEEE Transactions on Automatic Control.

[17]  Daniel Adelman,et al.  Relaxations of Weakly Coupled Stochastic Dynamic Programs , 2008, Oper. Res..

[18]  Steven H. Low,et al.  Convergence of Asynchronous Optimization Flow Control , 1999, Broadband Communications.

[19]  P. Varaiya.  Optimal and suboptimal stationary controls for Markov chains , 1978 .

[20]  Frank Kelly,et al.  Rate control for communication networks: shadow prices, proportional fairness and stability , 1998, J. Oper. Res. Soc..

[21]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[22]  P. Brémaud.  Point Processes and Queues , 1981 .

[23]  Yu-Chi Ho.  Review of the Witsenhausen problem , 2008, 2008 47th IEEE Conference on Decision and Control.

[24]  D. Luenberger.  Optimization by Vector Space Methods , 1968 .

[25]  Yu-Chi Ho,et al.  Team decision theory and information structures , 1980 .