Optimal Decentralized Control of Coupled Subsystems With Control Sharing

Subsystems that are coupled due to dynamics and costs arise naturally in various communication applications. In many such applications the control actions are shared between different control stations giving rise to a control sharing information structure. Previous studies of control-sharing have concentrated on the linear quadratic Gaussian setup and a solution approach tailored to continuous valued control actions. In this paper a three step solution approach for finite valued control actions is presented. In the first step, a person-by-person approach is used to identify redundant data or a sufficient statistic for local information at each control station. In the second step, the common-information based approach of Nayyar (2013) is used to find a sufficient statistic for the common information shared between all control stations and to obtain a dynamic programming decomposition. In the third step, the specifics of the model are used to simplify the sufficient statistic and the dynamic program.

[1]  Sanjay Lall,et al.  A Characterization of Convex Problems in Decentralized Control$^ast$ , 2005, IEEE Transactions on Automatic Control.

[2]  Tamer Basar,et al.  Stochastic Networked Control Systems: Stabilization and Optimization under Information Constraints , 2013 .

[3]  Gregory W. Wornell,et al.  A separation theorem for periodic sharing information patterns in decentralized control , 1997 .

[4]  Bruce E. Hajek,et al.  Paging and Registration in Cellular Networks: Jointly Optimal Policies and an Iterative Algorithm , 2007, IEEE Transactions on Information Theory.

[5]  Ashutosh Nayyar,et al.  Sequential Decision Making in Decentralized Systems , 2011 .

[6]  Yu-Chi Ho,et al.  Team decision theory and information structures , 1980 .

[7]  S. Marcus,et al.  Static team problems--Part I: Sufficient conditions and the exponential cost criterion , 1982 .

[8]  Serdar Yüksel,et al.  Stochastic Nestedness and the Belief Sharing Information Pattern , 2009, IEEE Transactions on Automatic Control.

[9]  O. Hernández-Lerma,et al.  Discrete-time Markov control processes , 1999 .

[10]  J. Bismut.  An example of interaction between information and control , 1973 .

[11]  François Charpillet,et al.  MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPs , 2005, UAI.

[12]  M. Athans,et al.  Solution of some nonclassical LQG stochastic decision problems , 1974 .

[13]  Pravin Varaiya,et al.  Stochastic Systems: Estimation, Identification, and Adaptive Control , 1986 .

[14]  Y. Ho,et al.  Team decision theory and information structures in optimal control problems--Part II , 1972 .

[15]  Jean C. Walrand,et al.  Optimal causal coding - decoding problems , 1983, IEEE Trans. Inf. Theory.

[16]  Aditya Mahajan.  Optimal transmission policies for two-user multiple access broadcast using dynamic team theory , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[17]  Robert G. Gallager,et al.  Multiaccess of a slotted channel by finitely many users , 1981 .

[18]  H. Vincent Poor,et al.  Decentralized Sequential Detection with a Fusion Center Performing the Sequential Test , 1992, 1992 American Control Conference.

[19]  Bruce Hajek,et al.  Decentralized dynamic control of a multiaccess broadcast channel , 1982 .

[20]  J. Walrand,et al.  On delayed sharing patterns , 1978 .

[21]  Onésimo Hernández-Lerma,et al.  Markov Control Processes , 1996 .

[22]  Hans S. Witsenhausen,et al.  A standard form for sequential stochastic control , 1973, Mathematical systems theory.

[23]  Sanjay Lall,et al.  A dynamic programming algorithm for decentralized Markov decision processes with a broadcast structure , 2010, 49th IEEE Conference on Decision and Control (CDC).

[24]  Petros G. Voulgaris,et al.  A convex characterization of distributed control problems in spatially invariant systems with communication constraints , 2005, Syst. Control. Lett..

[25]  Frits C. Schoute.  Decentralized control in packet switched satellite communication , 1978 .

[26]  Nikos A. Vlassis,et al.  Perseus: Randomized Point-based Value Iteration for POMDPs , 2005, J. Artif. Intell. Res..

[27]  R. Radner,et al.  Team Decision Problems , 1962 .

[28]  John Rust.  Using Randomization to Break the Curse of Dimensionality , 1997 .

[29]  Aditya Mahajan.  Optimal Decentralized Control of Coupled Subsystems With Control Sharing , 2013, IEEE Trans. Autom. Control..

[30]  François Charpillet,et al.  An Optimal Best-First Search Algorithm for Solving Infinite Horizon DEC-POMDPs , 2005, ECML.

[31]  H. Witsenhausen.  On the structure of real-time source coders , 1979, The Bell System Technical Journal.

[32]  Ashutosh Nayyar,et al.  Decentralized Stochastic Control with Partial Sharing Information Structures : A Common Information Approach , 2011 .

[33]  Demosthenis Teneketzis,et al.  Sequential decomposition of sequential dynamic teams: applications to real-time communication and networked control systems , 2008 .

[34]  Lois Kleffman,et al.  Over Time , 2014 .

[35]  D. Teneketzis,et al.  Identifying tractable decentralized control problems on the basis of information structure , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[36]  Ashutosh Nayyar,et al.  Optimal Control Strategies in Delayed Sharing Information Structures , 2010, IEEE Transactions on Automatic Control.

[37]  Aditya Mahajan,et al.  Measure and cost dependent properties of information structures , 2010, Proceedings of the 2010 American Control Conference.

[38]  G. W. Wornell,et al.  Decentralized control of a multiple access broadcast channel: performance bounds , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[39]  C. Striebel.  Sufficient statistics in the optimum control of stochastic systems , 1965 .

[40]  A. Rantzer.  Linear quadratic team theory revisited , 2006, 2006 American Control Conference.

[41]  Jean Walrand,et al.  Decentralized control in packet switched satellite communication , 1979 .

[42]  Shlomo Zilberstein,et al.  Memory-Bounded Dynamic Programming for DEC-POMDPs , 2007, IJCAI.

[43]  Ashutosh Nayyar,et al.  Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach , 2012, IEEE Transactions on Automatic Control.

[44]  Ather Gattami,et al.  Generalized Linear Quadratic Control , 2010, IEEE Transactions on Automatic Control.

[45]  M. Aicardi,et al.  Decentralized optimal control of Markov chains with a common past information set , 1987 .

[46]  Shlomo Zilberstein,et al.  Dynamic Programming for Partially Observable Stochastic Games , 2004, AAAI.

[47]  H. Witsenhausen.  Separation of estimation and control for discrete time systems , 1971 .

[48]  Michael I. Jordan,et al.  PEGASUS: A policy search method for large MDPs and POMDPs , 2000, UAI.