The Common-Information Approach to Decentralized Stochastic Control

Decentralized stochastic control arises in multi-stage decision-making with multiple decision-makers having different information and a common objective. Examples include cyber-physical systems, communication networks, sensing and surveillance systems, transportation systems, etc. In this chapter, we present the common-information approach to decentralized stochastic control. The key idea behind this approach is to formulate an equivalent centralized stochastic control problem from the point of view of a fictitious coordinator that observes only the information that is commonly available to all decision-makers. The optimal control problem for the fictitious coordinator is shown to be a partially observable Markov decision process (POMDP) which can be solved using techniques from Markov decision theory. We describe this approach for a general model and illustrate it by examples from real-time communication, networked control systems, paging and registration in cellular systems, and multi-access broadcast systems.

[1]  G. W. Wornell,et al.  Decentralized control of a multiple access broadcast channel: performance bounds , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[2]  Yu-Chi Ho,et al.  The Decentralized Wald Problem , 1987, Inf. Comput..

[3]  Hans S. Witsenhausen,et al.  A standard form for sequential stochastic control , 1973, Mathematical systems theory.

[4]  Tsuneo Yoshikawa.  Dynamic programming approach to decentralized stochastic control problems , 1974, CDC 1974.

[5]  Demosthenis Teneketzis.  The decentralized quickest detection problem , 1982, 1982 21st IEEE Conference on Decision and Control.

[6]  J. Bismut.  An example of interaction between information and control , 1973 .

[7]  Neri Merhav,et al.  Structure theorem for real-time variable-rate lossy source encoders and memory-limited decoders with side information , 2010, 2010 IEEE International Symposium on Information Theory.

[8]  Ming Cao,et al.  Proceedings of the 49th IEEE Conference on Decision and Control , 2010, IEEE Conference on Decision and Control.

[9]  Robert R. Tenney,et al.  Detection with distributed sensors , 1980 .

[10]  A. Rantzer.  Linear quadratic team theory revisited , 2006, 2006 American Control Conference.

[11]  Peter Whittle,et al.  Optimization Over Time , 1982 .

[12]  Jean C. Walrand,et al.  Optimal causal coding - decoding problems , 1983, IEEE Trans. Inf. Theory.

[13]  Ilya V. Kolmanovsky,et al.  Predictive energy management of a power-split hybrid electric vehicle , 2009, 2009 American Control Conference.

[14]  Sanjay Lall,et al.  A unifying condition for separable two player optimal control problems , 2011, IEEE Conference on Decision and Control and European Control Conference.

[15]  J. Walrand,et al.  On delayed sharing patterns , 1978 .

[16]  Sanjay Lall,et al.  A Characterization of Convex Problems in Decentralized Control$^ast$ , 2005, IEEE Transactions on Automatic Control.

[17]  Ashutosh Nayyar,et al.  Decentralized Detection with Signaling , 2010 .

[18]  Bruce E. Hajek,et al.  Paging and registration in cellular networks: jointly optimal policies and an iterative algorithm , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[19]  Petros G. Voulgaris,et al.  A convex characterization of distributed control problems in spatially invariant systems with communication constraints , 2005, Syst. Control. Lett..

[20]  Ashutosh Nayyar,et al.  Decentralized Stochastic Control with Partial Sharing Information Structures : A Common Information Approach , 2011 .

[21]  M. Aicardi,et al.  Decentralized optimal control of Markov chains with a common past information set , 1987 .

[22]  Frits C. Schoute.  Decentralized control in packet switched satellite communication , 1978 .

[23]  Sanjay Lall,et al.  A dynamic programming algorithm for decentralized Markov decision processes with a broadcast structure , 2010, 49th IEEE Conference on Decision and Control (CDC).

[24]  H. Vincent Poor,et al.  Decentralized sequential detection with sensors performing sequential tests , 1994, Math. Control. Signals Syst..

[25]  J. Tsitsiklis.  Decentralized Detection' , 1993 .

[26]  H. Witsenhausen.  On the structure of real-time source coders , 1979, The Bell System Technical Journal.

[27]  Ashutosh Nayyar,et al.  Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach , 2012, IEEE Transactions on Automatic Control.

[28]  Ather Gattami.  Control and estimation problems under partially nested information pattern , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[29]  H. Vincent Poor,et al.  Decentralized Sequential Detection with a Fusion Center Performing the Sequential Test , 1992 .

[30]  Ashutosh Nayyar,et al.  On globally optimal real-time encoding and decoding strategies in multi-terminal communication systems , 2008, 2008 47th IEEE Conference on Decision and Control.

[31]  Demosthenis Teneketzis,et al.  On the Structure of Optimal Real-Time Encoders and Decoders in Noisy Communication , 2006, IEEE Transactions on Information Theory.

[32]  Yu-Chi Ho,et al.  Team decision theory and information structures , 1980 .

[33]  Serdar Yüksel,et al.  Stochastic Nestedness and the Belief Sharing Information Pattern , 2009, IEEE Transactions on Automatic Control.

[34]  Ashutosh Nayyar,et al.  Optimal Control Strategies in Delayed Sharing Information Structures , 2010, IEEE Transactions on Automatic Control.

[35]  Demosthenis Teneketzis,et al.  On the design of globally optimal communication strategies for real-time noisy communication systems with noisy feedback , 2008, IEEE Journal on Selected Areas in Communications.

[36]  Y. Ho,et al.  Team decision theory and information structures in optimal control problems--Part II , 1972 .

[37]  H. Witsenhausen.  Separation of estimation and control for discrete time systems , 1971 .

[38]  Robert G. Gallager,et al.  Multiaccess of a slotted channel by finitely many users , 1981 .

[39]  Sanjay Lall,et al.  A state-space solution to the two-player decentralized optimal control problem , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[40]  Aditya Mahajan,et al.  Optimal Decentralized Control of Coupled Subsystems With Control Sharing , 2011, IEEE Transactions on Automatic Control.

[41]  Venugopal V. Veeravalli.  Decentralized quickest change detection , 2001, IEEE Trans. Inf. Theory.

[42]  Hao Zhang,et al.  Partially Observable Markov Decision Processes: A Geometric Technique and Analysis , 2010, Oper. Res..

[43]  D. Teneketzis,et al.  Identifying tractable decentralized control problems on the basis of information structure , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[44]  Jean Walrand,et al.  Causal coding and control for Markov chains , 1983 .

[45]  Gregory W. Wornell,et al.  A separation theorem for periodic sharing information patterns in decentralized control , 1997 .

[46]  Ashutosh Nayyar,et al.  Sequential Decision Making in Decentralized Systems , 2011 .