Centralized Versus Decentralized Optimization of Distributed Stochastic Differential Decision Systems With Different Information Structures-Part I: A General Theory

Decentralized optimization of distributed stochastic dynamical systems with two or more controls of the decision makers (DMs) has been an active area of research for over half a century. Although, such decentralized optimization problems are often formulated utilizing static team and person-by-person (PbP) optimality criteria, the corresponding static team theory results have not been extended to dynamical systems. In this first part of the two-part paper, we derive team and PbP optimality conditions for distributed stochastic differential systems, when the controls of the DMs generate actions based on different information structures. The necessary conditions are given by a Hamiltonian System described by coupled backward and forward stochastic differential equations (SDEs) and a conditional Hamiltonian, conditioned on the information structures available to the controls of the DMs. The sufficient conditions state that PbP optimality implies team optimality, if the Hamiltonian is convex in the state and/or actions spaces of the controls of the DMs. We show existence of relaxed team optimal strategies, when the information structures are not affected by the controls of the DMs. Throughout the paper we discuss similarities to analogous optimality conditions of centralized decision or control of stochastic systems, and we note a connections to mean field stochastic optimal control problems.

[1]  Nuno C. Martins,et al.  Information structures in optimal decentralized control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[2]  Yu-Chi Ho,et al.  Team decision theory and information structures , 1980 .

[3]  R. Sivan,et al.  Linear-quadratic-gaussian control with one-step-delay sharing pattern , 1974 .

[4]  S. Marcus,et al.  Static team problems--Part I: Sufficient conditions and the exponential cost criterion , 1982 .

[5]  Tsuneo Yoshikawa,et al.  Dynamic programming approach to decentralized stochastic control problems , 1974, CDC 1974.

[6]  Demosthenis Teneketzis,et al.  Optimal Performance of Networked Control Systems with Nonclassical Information Structures , 2009, SIAM J. Control. Optim..

[7]  B. Kurtaran.  A concise derivation of the LQG one-step-delay sharing problem solution , 1975 .

[8]  P. Caines,et al.  Social optima in mean field LQG control: Centralized and decentralized strategies , 2009 .

[9]  Yu-Chi Ho,et al.  On the equivalence of information structures in static and dynamic teams , 1973 .

[10]  A. Bensoussan.  Stochastic Control of Partially Observable Systems , 1992 .

[11]  M. Aicardi,et al.  Decentralized optimal control of Markov chains with a common past information set , 1987 .

[12]  N. U. Ahmed,et al.  OPTIMAL RELAXED CONTROLS FOR SYSTEMS GOVERNED BY IMPULSIVE DIFFERENTIAL INCLUSIONS , 2016 .

[13]  Ashutosh Nayyar,et al.  Optimal Control Strategies in Delayed Sharing Information Structures , 2010, IEEE Transactions on Automatic Control.

[14]  C. Ionescu Tulcea,et al.  Topics in the Theory of Lifting , 1969 .

[15]  J. Bismut.  An Introductory Approach to Duality in Optimal Stochastic Control , 1978 .

[16]  Nasir Uddin Ahmed,et al.  Minimax games for stochastic systems subject to relative entropy uncertainty: applications to SDEs on Hilbert spaces , 2007, Math. Control. Signals Syst..

[17]  Jan Komenda,et al.  Control of Distributed Systems: Tutorial and Overview , 2011, Eur. J. Control.

[18]  Nasir Uddin Ahmed,et al.  Centralized Versus Decentralized Team Games of Distributed Stochastic Differential Decision Systems with Noiseless Information Structures-Part II: Applications , 2013, ArXiv.

[19]  J. Marschak,et al.  Elements for a Theory of Teams , 1955 .

[20]  Y. Ho,et al.  Team decision theory and information structures in optimal control problems--Part II , 1972 .

[21]  J. Walrand,et al.  On delayed sharing patterns , 1978 .

[22]  H. Witsenhausen.  A Counterexample in Stochastic Optimum Control , 1968 .

[23]  Robert J. Elliott,et al.  The second order minimum principle and adjoint process , 1994 .

[24]  R. Radner,et al.  Team Decision Problems , 1962 .

[25]  Robert J. Elliott.  The Optimal Control of a Stochastic System , 1977 .

[26]  Alain Bensoussan,et al.  Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions , 1983 .

[27]  Ashutosh Nayyar,et al.  Decentralized Stochastic Control with Partial History Sharing: A Common Information Approach , 2012, IEEE Transactions on Automatic Control.

[28]  Tamer Basar,et al.  Team decision theory for linear continuous-time systems , 1980 .

[29]  Charalambos D. Charalambous.  A sufficient condition for decentralized non-cooperative stochastic differential games and relations to mean field games , 2015, 2015 54th IEEE Conference on Decision and Control (CDC).

[30]  Demosthenis Teneketzis,et al.  Optimal Design of Sequential Real-Time Communication Systems , 2009, IEEE Transactions on Information Theory.

[31]  N. U. Ahmed,et al.  Linear and Nonlinear Filtering for Scientists and Engineers , 1999 .

[32]  Robert J. Elliott,et al.  Control of partially observed diffusions , 1991 .

[33]  V. Benes.  Existence of Optimal Stochastic Control Laws , 1971 .

[34]  Nasir Uddin Ahmed,et al.  Stochastic Minimum Principle for Partially Observed Systems Subject to Continuous and Jump Diffusion Processes and Driven by Relaxed Controls , 2013, SIAM J. Control. Optim..

[35]  Jan H. van Schuppen,et al.  A Class of Team Problems with Discrete Action Spaces: Optimality Conditions Based on Multimodularity , 2000, SIAM J. Control. Optim..

[36]  Kok Lay Teo,et al.  Optimal control of distributed parameter systems , 1981 .

[37]  Jan H. van Schuppen,et al.  Control of Distributed Stochastic Systems – Introduction, Problems, and Approaches , 2011 .

[38]  T. Başar,et al.  Stochastic Teams with Nonclassical Information Revisited: When is an Affine Law Optimal? , 1986, 1986 American Control Conference.

[39]  H. Witsenhausen.  Separation of estimation and control for discrete time systems , 1971 .

[40]  Bohdan Maslowski,et al.  A stochastic maximum principle for optimal control of diffusions , 1988, Acta Applicandae Mathematicae.

[41]  C. Charalambous,et al.  Minimum principle for partially observable nonlinear risk-sensitive control problems using measure-valued decompositions , 1996 .

[42]  Nasir Uddin Ahmed,et al.  Centralized Versus Decentralized Team Games of Distributed Stochastic Differential Decision Systems with Noiseless Information Structures-Part I: General Theory , 2013, ArXiv.

[43]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[44]  Steven I. Marcus,et al.  Static team problems--Part II: Affine control laws, projections, algorithms, and the LEGT problem , 1982 .

[45]  Hans S. Witsenhausen,et al.  Equivalent stochastic control problems , 1988, Math. Control. Signals Syst..

[46]  Petros G. Voulgaris,et al.  A convex characterization of distributed control problems in spatially invariant systems with communication constraints , 2005, Syst. Control. Lett..