Optimal Control for Continuous-Time Linear Quadratic Problems with Infinite Markov Jump Parameters

The subject matter of this paper is the optimal control problem for continuous-time linear systems subject to Markovian jumps in the parameters and the usual infinite-time horizon quadratic cost. What essentially distinguishes our problem from previous ones, inter alia, is that the Markov chain takes values on a countably infinite set. To tackle our problem, we make use of powerful tools from semigroup theory in Banach space and a decomplexification technique. The solution for the problem relies, in part, on the study of a countably infinite set of coupled algebraic Riccati equations (ICARE). Conditions for existence and uniqueness of a positive semidefinite solution of the ICARE are obtained via the extended concepts of stochastic stabilizability (SS) and stochastic detectability (SD). These concepts are couched into the theory of operators in Banach space and, parallel to the classical linear quadratic (LQ) case, bound up with the spectrum of a certain infinite dimensional linear operator.

[1]  K. Loparo,et al.  Stochastic stability properties of jump linear systems , 1992 .

[2]  L. Arnold.  Stochastic Differential Equations: Theory and Applications , 1992 .

[3]  E. M. Hemerly,et al.  Optimal control for a class of noisy linear systems with markovian jumping parameters and quadratic cost , 1991 .

[4]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[5]  Marcelo D. Fragoso,et al.  A Small Random Perturbation Analysis of a Partially Observable LQG Problem for Systems with Markovian Jumping Parameters , 1990 .

[6]  Kim C. Border,et al.  Infinite Dimensional Analysis: A Hitchhiker’s Guide , 1994 .

[7]  W. Fleming,et al.  Deterministic and Stochastic Optimal Control , 1975 .

[8]  A. Krall.  Applied Analysis , 1986 .

[9]  M. Fragoso,et al.  On a partially observable LQG problem for systems with Markovian jumping parameters , 1988 .

[10]  R. Rogers,et al.  An LQ-solution to a control problem associated with a solar thermal central receiver , 1983 .

[11]  S. Karlin,et al.  A second course in stochastic processes , 1981 .

[12]  V. Arnold,et al.  Ordinary Differential Equations , 1973 .

[13]  M. Mariton,et al.  Jump Linear Systems in Automatic Control , 1992 .

[14]  M. Fragoso,et al.  Stability Results for Discrete-Time Linear Systems with Markovian Jumping Parameters , 1993 .

[15]  H. McKean,et al.  Diffusion processes and their sample paths , 1996 .

[16]  Amnon Pazy,et al.  Semigroups of Linear Operators and Applications to Partial Differential Equations , 1992, Applied Mathematical Sciences.

[17]  W.S. Gray,et al.  Modeling electromagnetic disturbances in closed-loop computer controlled flight systems , 1998, Proceedings of the 1998 American Control Conference. ACC (IEEE Cat. No.98CH36207).

[18]  R. Curtain.  A semigroup approach to the LQG problem for infinite-dimensional systems , 1978 .

[19]  R. P. Marques,et al.  Mixed H2/H∞-control of discrete-time Markovian jump linear systems , 1998, IEEE Trans. Autom. Control..

[20]  Arch W. Naylor,et al.  Linear Operator Theory in Engineering and Science , 1971 .

[21]  El-Kébir Boukas,et al.  Stochastic stability and guaranteed cost control of discrete-time uncertain systems with Markovian jumping parameters , 1997 .

[22]  Tamer Basar,et al.  Receding horizon control of jump linear systems and a macroeconomic policy problem , 1999 .

[23]  M. Mariton,et al.  Output feedback for a class of linear systems with stochastic jump parameters , 1985 .

[24]  R. Elliott,et al.  Adaptive control of linear systems with Markov perturbations , 1998, IEEE Trans. Autom. Control..

[25]  M. Fragoso,et al.  Discrete-time LQ-optimal control problems for infinite Markov jump parameter systems , 1995, IEEE Trans. Autom. Control..

[26]  T. Morozan,et al.  Optimal stationary control for dynamic systems with Markov perturbations , 1983 .

[27]  M. Fragoso,et al.  Optimal control for continuous time LQ-problems with infinite Markov jump parameters via semigroup , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[28]  J. Hiriart-Urruty,et al.  Convex analysis and minimization algorithms , 1993 .

[29]  M. Fragoso,et al.  On an infinite dimensional perturbed Riccati differential equation arising in stochastic control , 2001, 2001 European Control Conference (ECC).

[30]  Robert J. Elliott,et al.  Control of a hybrid conditionally linear Gaussian process , 1992 .

[31]  Michel Mariton,et al.  Almost sure and moments stability of jump linear systems , 1988 .

[32]  François Dufour,et al.  The filtering problem for continuous-time linear systems with Markovian switching coefficients , 1994 .

[33]  W. Wonham.  On a Matrix Riccati Equation of Stochastic Control , 1968 .