State Distributions and Minimum Relative Entropy Noise Sequences in Uncertain Stochastic Systems: The Discrete-Time Case

The paper is concerned with a dissipativity theory and robust performance analysis of discrete-time stochastic systems driven by a statistically uncertain random noise. The uncertainty is quantified by the conditional relative entropy of the actual probability law of the noise with respect to a nominal product measure corresponding to a white noise sequence. We discuss a balance equation, dissipation inequality and superadditivity property for the corresponding conditional relative entropy supply as a function of time. The problem of minimizing the supply required to drive the system between given state distributions over a specified time horizon is considered. Such variational problems, involving entropy and probabilistic boundary conditions, are known in the literature as Schroedinger bridge problems. In application to control systems, this minimum required conditional relative entropy supply characterizes the robustness of the system with respect to an uncertain noise. We obtain a dynamic programming Bellman equation for the minimum required conditional relative entropy supply and establish a Markov property of the worst-case noise with respect to the state of the system. For multivariable linear systems with a Gaussian white noise sequence as the nominal noise model and Gaussian initial and terminal state distributions, the minimum required supply is obtained using an algebraic Riccati equation which admits a closed-form solution. We propose a computable robustness index for such systems in the framework of an entropy theoretic formulation of uncertainty and provide an example to illustrate this approach.

[1]  R. Gray Entropy and Information Theory , 1990, Springer New York.

[2]  O. Gaans Probability measures on metric spaces , 2022 .

[3]  Leiba Rodman,et al.  Algebraic Riccati equations , 1995 .

[4]  A. P. Kurdjukov,et al.  Anisotropy-based performance analysis of linear discrete time invariant control systems , 2001 .

[5]  Ian R. Petersen,et al.  Minimum Relative Entropy State Transitions in Linear Stochastic Systems: the Continuous Time Case , 2012, ArXiv.

[6]  J. Willems Dissipative dynamical systems part I: General theory , 1972 .

[7]  Toshio Mikami,et al.  Variational processes from the weak forward equation , 1990 .

[8]  Ian R. Petersen,et al.  MINIMAX LQG CONTROL , 2006 .

[9]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[10]  S. Taylor,et al.  INTEGRAL, MEASURE AND DERIVATIVE , 1967 .

[11]  Peter E. Kloeden,et al.  Anisotropy-based robust performance analysis of finite horizon linear discrete time varying systems , 2006 .

[12]  A. P. Kurdjukov,et al.  On Computing the Anisotropic Norm of Linear Discrete-Time-Invariant Systems , 1996 .

[13]  W. Rudin Real and complex analysis, 3rd ed. , 1987 .

[14]  I. Postlethwaite,et al.  State-space formulae for discrete-time H∞ optimization , 1989 .

[15]  N. Martin,et al.  Mathematical Theory of Entropy , 1981 .

[16]  A. Beghi,et al.  On the relative entropy of discrete-time Markov processes with given end-point densities , 1996, IEEE Trans. Inf. Theory.

[17]  Michele Pavon,et al.  How to Steer a Quantum System over a Schrödinger Bridge , 2002, Quantum Inf. Process..

[18]  Huibert Kwakernaak,et al.  Linear Optimal Control Systems , 1972 .

[19]  A. P. Kurdjukov,et al.  State-Space Solution to Anisotropy-Based Stochastic H∞-Optimization Problem , 1996 .

[20]  Li Xie,et al.  Finite horizon robust state estimation for uncertain finite-alphabet hidden Markov models with conditional relative entropy constraints , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[21]  J. Lynch,et al.  A weak convergence approach to the theory of large deviations , 1997 .

[22]  M. Aschwanden Statistics of Random Processes , 2021, Biomedical Measurement Systems and Data Science.

[23]  Ian R. Petersen,et al.  Robust Properties of Risk-Sensitive Control , 2000, Math. Control. Signals Syst..

[24]  M. Mariton,et al.  A homotopy algorithm for solving coupled riccati equations , 1985 .

[25]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Statistics of random processes , 1977 .

[26]  R. Schilling Measures, Integrals and Martingales: Frontmatter , 2006 .

[27]  In Choi,et al.  TIME-SERIES-BASED ECONOMETRICS , 1998, Econometric Theory.

[28]  R. Xu,et al.  Theory of open quantum systems , 2002 .

[29]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[30]  Ian R. Petersen,et al.  Minimax LQG Control of Stochastic Partially Observed Uncertain Systems , 2001, SIAM J. Control. Optim..

[31]  Igor G. Vladimirov,et al.  Anisotropic Norm Bounded Real Lemma for Linear Discrete Time Varying Systems , 2012, ArXiv.

[32]  Michele Pavon,et al.  Schr ¨ odinger bridges for discrete-time, classical and quantum Markovian evolutions , 2010 .

[33]  A. P. Kurdjukov,et al.  Stochastic approach to H/sub /spl infin//-optimization , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[34]  W. Rudin Real and complex analysis , 1968 .

[35]  Karolos M. Grigoriadis,et al.  A unified algebraic approach to linear control design , 1998 .

[36]  I. Petersen,et al.  Robust Control Design Using H-? Methods , 2012 .

[37]  Charalambos D. Charalambous,et al.  Stochastic Uncertain Systems Subject to Relative Entropy Constraints: Induced Norms and Monotonicity Properties of Minimax Games , 2007, IEEE Transactions on Automatic Control.

[38]  B. Anderson,et al.  Optimal Filtering , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  A. Blaquière,et al.  Controllability of a Fokker-Planck equation, the Schrödinger system, and a related stochastic optimal control (revised version) , 1992 .

[40]  Ian R. Petersen,et al.  Finite horizon robust state estimation for uncertain finite-alphabet hidden Markov models with conditional relative entropy constraints , 2004, CDC.

[41]  Edward Nelson Dynamical Theories of Brownian Motion , 1967 .

[42]  Michele Pavon,et al.  On the Geometry of Maximum Entropy Problems , 2011, SIAM Rev..

[43]  Jan C. Willems,et al.  Dissipative Dynamical Systems , 2007, Eur. J. Control.

[44]  D. Naidu,et al.  Optimal Control Systems , 2018 .

[45]  Ian R. Petersen,et al.  Minimum relative entropy state transitions in discrete time systems with statistically uncertain noise , 2010, 49th IEEE Conference on Decision and Control (CDC).

[46]  K. Glover,et al.  State-space approach to discrete-time H∞ control , 1991 .

[47]  J. Willems Dissipative dynamical systems Part II: Linear systems with quadratic supply rates , 1972 .

[48]  Paolo Dai Pra,et al.  A stochastic control approach to reciprocal diffusion processes , 1991 .

[49]  Alessandro Beghi,et al.  Continuous-time Gauss-Markov processes with fixed reciprocal dynamics , 1997 .

[50]  Richard L. Tweedie,et al.  Markov Chains and Stochastic Stability , 1993, Communications and Control Engineering Series.

[51]  Ian R. Petersen,et al.  Minimax optimal control of stochastic uncertain systems with relative entropy constraints , 2000, IEEE Trans. Autom. Control..

[52]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[53]  Ian R. Petersen,et al.  Robust Control Design Using H-infinity Methods , 2000 .

[54]  N. Higham Functions Of Matrices , 2008 .