Stochastic and adaptive optimal control of uncertain interconnected systems: A data-driven approach

Abstract This paper provides a novel non-model-based, data-driven stochastic H ∞ control design for linear continuous-time stochastic interconnected systems with unknown dynamics. Our contributions are three-fold. First, we develop a tool to show how to assign an arbitrarily small input-to-output stochastic L 2 gain of the closed-loop system, by combining the gain assignment technique with the zero-sum dynamic game-based H ∞ control design. Second, robustness to dynamic uncertainties is tackled using the small-gain theory. Third, we develop a non-model-based stochastic robust adaptive dynamic programming (RADP) algorithm for adaptive optimal controller design. In sharp contrast to the existing methods, the obtained algorithm is based on value iteration (VI), and the knowledge of an initial stabilizing control policy is no longer needed. An example of a power electronic system is adopted to illustrate the obtained results.

[1]  A. Mesbah,et al.  Stochastic Model Predictive Control: An Overview and Perspectives for Future Research , 2016, IEEE Control Systems.

[2]  Zhong-Ping Jiang,et al.  Adaptive dynamic programming and optimal control of nonlinear nonaffine systems , 2014, Autom..

[3]  Jan C. Willems,et al.  Feedback stabilizability for stochastic systems with state and control dependent noise , 1976, Autom..

[4]  Michael P. Wellman,et al.  Nash Q-Learning for General-Sum Stochastic Games , 2003, J. Mach. Learn. Res..

[5]  Zhong-Ping Jiang,et al.  A Tool for the Global Stabilization of Stochastic Nonlinear Systems , 2017, IEEE Transactions on Automatic Control.

[6]  Zhong-Ping Jiang,et al.  A small-gain control method for nonlinear cascaded systems with dynamic uncertainties , 1997, IEEE Trans. Autom. Control..

[7]  Kyriakos G. Vamvoudakis,et al.  Non-zero sum Nash Q-learning for unknown deterministic continuous-time linear systems , 2015, Autom..

[8]  D. Kleinman,et al.  Optimal stationary control of linear systems with control-dependent noise , 1969 .

[9]  Yu Jiang,et al.  Robust Adaptive Dynamic Programming and Feedback Stabilization of Nonlinear Systems , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Robert W. Erickson,et al.  Fundamentals of Power Electronics , 2001 .

[11]  Zhong-Ping Jiang,et al.  Value iteration and adaptive dynamic programming for data-driven adaptive optimal control design , 2016, Autom..

[12]  Haibo He,et al.  Model-Free Dual Heuristic Dynamic Programming , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[13]  Zhong-Ping Jiang,et al.  Adaptive dynamic programming as a theory of sensorimotor control , 2012, 2012 IEEE Signal Processing in Medicine and Biology Symposium (SPMB).

[14]  Xi Chen,et al.  Solvability and asymptotic behavior of generalized Riccati equations arising in indefinite stochastic LQ controls , 2001, IEEE Trans. Autom. Control..

[15]  Ian R. Petersen,et al.  Absolute Stabilization and Minimax Optimal Control of Uncertain Systems with Stochastic Uncertainty , 1997 .

[16]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[17]  D. Hinrichsen,et al.  Stochastic $H^\infty$ , 1998 .

[18]  Warren E. Dixon,et al.  Concurrent learning-based approximate feedback-Nash equilibrium solution of N-player nonzero-sum differential games , 2013, IEEE/CAA Journal of Automatica Sinica.

[19]  L. Arnold Stochastic Differential Equations: Theory and Applications , 1992 .

[20]  Zhong-Ping Jiang,et al.  Decentralized Adaptive Optimal Control of Large-Scale Systems With Application to Power Systems , 2015, IEEE Transactions on Industrial Electronics.

[21]  Brian D. O. Anderson,et al.  Computing the Positive Stabilizing Solution to Algebraic Riccati Equations With an Indefinite Quadratic Term via a Recursive Method , 2008, IEEE Transactions on Automatic Control.

[22]  T. Basar,et al.  H∞-0ptimal Control and Related Minimax Design Problems: A Dynamic Game Approach , 1996, IEEE Trans. Autom. Control..

[23]  Zhong-Ping Jiang,et al.  Decentralized adaptive output-feedback stabilization for large-scale stochastic nonlinear systems , 2007, Autom..

[24]  Zhong-Ping Jiang,et al.  Robust adaptive dynamic programming for linear and nonlinear systems: An overview , 2013, Eur. J. Control.

[25]  Zhong-Ping Jiang,et al.  Small-gain theorem for ISS systems and applications , 1994, Math. Control. Signals Syst..

[26]  Yuan Wang,et al.  Stabilization in spite of matched unmodeled dynamics and an equivalent definition of input-to-state stability , 1996, Math. Control. Signals Syst..

[27]  William B. Haskell,et al.  Empirical Dynamic Programming , 2013, Math. Oper. Res..

[28]  Miroslav Krstic,et al.  Stabilization of Nonlinear Uncertain Systems , 1998 .

[29]  Lei Guo,et al.  Adaptive continuous-time linear quadratic Gaussian control , 1999, IEEE Trans. Autom. Control..

[30]  T. Damm Rational Matrix Equations in Stochastic Control , 2004 .

[31]  V. Dragan,et al.  Mathematical Methods in Robust Control of Linear Stochastic Systems , 2006 .

[32]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[33]  Xue-Jun Xie,et al.  Adaptive backstepping controller design using stochastic small-gain theorem , 2007, Autom..

[34]  I. Petersen,et al.  Robust Control Design Using H-? Methods , 2012 .

[35]  Yungang Liu,et al.  Global stabilization via time-varying output-feedback for stochastic nonlinear systems with unknown growth rate , 2015, Syst. Control. Lett..