Control of Discrete-Time Hybrid Stochastic Systems

A realistic stochastic control problem for hybrid systems with Markovian jump parameters can have the switching parameters in both the state and measurement equations. Furthermore, both the system state and the jump states are, in general, not perfectly observed. Currently there are only two existing controllers for this problem. One is based upon a heuristic multiple model partitioning (MMP) and hypothesis pruning. The other utilizes the entire future tree of models, and is called the Full-Tree (FT) controller. The performance of the latter is significantly superior to the former and their complexities are similar. In this paper we present a new stochastic control algorithm for stochastic systems with Markovian jump parameters. This control algorithm is derived through the use of stochastic dynamic programming and is designed to be used for realistic stochastic control problems, i.e., with noisy state observations. This new scheme, which is based upon the interaction of r (the number of models) model-conditioned Riccati equations, has a natural parallel to implementation. The state estimation and model identification is done via the recently developed Interacting Multiple Model algorithm. Simulation results show that a substantial reduction in cost can be obtained by this new control algorithm over the MMP scheme. Furthermore, the performance of the new algorithm is shown to be practically the same as that of the FT scheme even though the new scheme, which has a fixed amount of computations at each step of the recursion, is much simpler than both the MMP and FT algorithms.

[1]  Eyad H. Abed,et al.  Guardian maps and the generalized stability of parametrized families of matrices and polynomials , 1990, Math. Control. Signals Syst..

[2]  Shaun Gutman,et al.  Root Clustering in Parameter Space , 1990 .

[3]  N. Bose.  Tests for Hurwitz and Schur properties of convex combination of complex polynomials , 1989 .

[4]  S. Bhattacharyya,et al.  A generalization of Kharitonov's theorem; Robust stability of interval plants , 1989 .

[5]  Y. Bar-Shalom,et al.  Tracking a maneuvering target using input estimation versus the interacting multiple model algorithm , 1989 .

[6]  M. Mariton,et al.  On systems with non-Markovian regime changes , 1989 .

[7]  Engin Yaz.  Stabilizing control of discrete systems with jump Markov parameters , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[8]  A. Haddad,et al.  Error bounds in the averaging of hybrid systems , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[9]  Peter S. Maybeck,et al.  Space structure control using moving bank multiple model adaptive estimation , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[10]  B. R. Barmish,et al.  Robust Schur stability of a polytope of polynomials , 1988 .

[11]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[12]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[13]  M. Mariton.  On the influence of noise on jump linear systems , 1987 .

[14]  Peter Maybeck,et al.  Moving bank multiple model adaptive estimation applied to flexible spacestructure control , 1987, 26th IEEE Conference on Decision and Control.

[15]  Huang Lin,et al.  Root locations of an entire polytope of polynomials: It suffices to check the edges , 1987, 1987 American Control Conference.

[16]  M. Mariton.  Jump Linear Quadratic control with Random state discontinuities , 1987, Autom..

[17]  David D. Sworder,et al.  Feedforward/Feedback Controls in a Noisy Environment , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[18]  A. Willsky,et al.  Discrete-time Markovian-jump linear quadratic optimal control , 1986 .

[19]  Kenneth Loparo,et al.  A dual controller for linear systems with random jump parameters , 1985, 1985 24th IEEE Conference on Decision and Control.

[20]  Barry E. Griffiths,et al.  Optimal control of jump-linear gaussian systems† , 1985 .

[21]  Control of Stochastic Systems with Markov Interrupted Observations , 1983, IEEE Transactions on Aerospace and Electronic Systems.

[22]  Y. Bar-Shalom,et al.  A multiple model adaptive dual control algorithm for stochastic systems with unknown parameters , 1979, 1979 18th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[23]  H. F. Vanlandingham,et al.  Digital Set Point Control of Nonlinear Stochastic Systems , 1978, IEEE Transactions on Industrial Electronics and Control Instrumentation.

[24]  R. Moose,et al.  Modeling and control of nonlinear plants , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[25]  D. Castañón,et al.  On reliable control system designs with and without feedback reconfigurations , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[26]  Michael Athans,et al.  The stochastic control of the F-8C aircraft using a multiple model adaptive control (MMAC) method--Part I: Equilibrium flight , 1977 .

[27]  H. Van Landingham,et al.  Digital Control of High Performance Aircraft Using Adaptive Estimation Techniques , 1977, IEEE Transactions on Aerospace and Electronic Systems.

[28]  Chelsea C. White,et al.  Finite-state, discrete-time optimization with randomly varying observation quality , 1976, Autom..

[29]  D.G. Lainiotis,et al.  Partitioning: A unifying framework for adaptive systems, II: Control , 1976, Proceedings of the IEEE.

[30]  D. Lainiotis,et al.  Partitioning: A unifying framework for adaptive systems, I: Estimation , 1976, Proceedings of the IEEE.

[31]  D.D. Sworder,et al.  Control of systems subject to sudden change in character , 1976, Proceedings of the IEEE.

[32]  V. V. Krishnan,et al.  Inners and Stability of Dynamic Systems , 1975, IEEE Transactions on Systems, Man, and Cybernetics.

[33]  Y. Bar-Shalom,et al.  Dual effect, certainty equivalence, and separation in stochastic control , 1974 .

[34]  George N. Saridis,et al.  A learning approach to the parameter-adaptive self-organizing control problem , 1972 .