An off‐policy approach for model‐free stabilization of linear systems subject to input energy constraint and its application to spacecraft rendezvous

This note is concerned with the problem of stabilizing a class of linear continuous‐time systems with completely unknown system dynamics subject to input energy constraint. To deal with this problem, a model‐based low gain feedback law is designed firstly by establishing a special algebraic Riccati equation. Such a low gain feedback law can semiglobally stabilize the linear systems subject to input energy constraint with the exact system model. In order to relax the assumption that the system model is exactly known, an off‐policy reinforcement learning approach is designed to solve the same problem without requiring the completely knowledge of the system dynamics. Finally, in order to verify the effectiveness of the proposed model‐free approach, simulation result on the spacecraft rendezvous problem is introduced.

[1]  Frank L. Lewis,et al.  Data-Driven Flotation Industrial Process Operational Optimal Control Based on Reinforcement Learning , 2018, IEEE Transactions on Industrial Informatics.

[2]  Frank L. Lewis,et al.  Dual-Rate Operational Optimal Control for Flotation Industrial Process With Unknown Operational Model , 2019, IEEE Transactions on Industrial Electronics.

[3]  Feng Liu,et al.  Online Supplementary ADP Learning Controller Design and Application to Power System Frequency Control With Large-Scale Wind Energy Integration , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Guang-Ren Duan,et al.  A Parametric Lyapunov Equation Approach to the Design of Low Gain Feedback , 2008, IEEE Transactions on Automatic Control.

[5]  Yong-Nong Chang,et al.  Robust anti-windup controller design of time-delay fuzzy systems with actuator saturations , 2011, Inf. Sci..

[6]  Kai Zhang,et al.  Adaptive Optimal Control With Guaranteed Convergence Rate for Continuous-Time Linear Systems With Completely Unknown Dynamics , 2019, IEEE Access.

[7]  Guang-Ren Duan,et al.  A parametric periodic Lyapunov equation with application in semi-global stabilization of discrete-time periodic systems subject to actuator saturation , 2010, Proceedings of the 2010 American Control Conference.

[8]  A. Saberi,et al.  Semi-global stabilization of linear discrete-time systems subject to input saturation via linear feedback-an ARE-based approach , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[9]  G. Duan,et al.  Circular orbital rendezvous with actuator saturation and delay: A parametric Lyapunov equation approach , 2012 .

[10]  Zhong-Ping Jiang,et al.  Data-Driven Adaptive Optimal Control of Connected Vehicles , 2017, IEEE Transactions on Intelligent Transportation Systems.

[11]  Guang-Ren Duan,et al.  A parametric Lyapunov equation approach to low gain feedback design for discrete-time systems , 2009, Autom..

[12]  Zhanshan Wang,et al.  Data-Based Optimal Control of Multiagent Systems: A Reinforcement Learning Design Approach , 2017, IEEE Transactions on Cybernetics.

[13]  Zongli Lin,et al.  Semi-global Exponential Stabilization of Linear Systems Subject to \input Saturation" via Linear Feedbacks , 1993 .

[14]  Yuanqing Xia,et al.  Adaptive Sliding Mode Control for Attitude Stabilization With Actuator Saturation , 2011, IEEE Transactions on Industrial Electronics.

[15]  Tianyou Chai,et al.  Optimal Output Regulation of Linear Discrete-Time Systems With Unknown Dynamics Using Reinforcement Learning , 2020, IEEE Transactions on Cybernetics.

[16]  Feng Liu,et al.  Approximate dynamic programming based supplementary reactive power control for DFIG wind farm to enhance power system stability , 2015, Neurocomputing.

[17]  Bin Zhou,et al.  On semi-global stabilization of linear periodic systems with control magnitude and energy saturations , 2015, J. Frankl. Inst..

[18]  Guang-Ren Duan,et al.  Periodic Lyapunov Equation Based Approaches to the Stabilization of Continuous-Time Periodic Linear Systems , 2012, IEEE Transactions on Automatic Control.

[19]  Frank L. Lewis,et al.  Model-Free Optimal Output Regulation for Linear Discrete-Time Lossy Networked Control Systems , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[20]  Chun Zhang,et al.  Adaptive reaching law based sliding mode control for electromagnetic formation flight with input saturation , 2016, J. Frankl. Inst..

[21]  Chunyu Yang,et al.  Anti-windup controller design for singularly perturbed systems subject to actuator saturation , 2016 .

[22]  Tao Li,et al.  Adaptive Dynamic Programming for Multi-intersections Traffic Signal Intelligent Control , 2008, 2008 11th International IEEE Conference on Intelligent Transportation Systems.

[23]  Frank L. Lewis,et al.  Tracking Control for Linear Discrete-Time Networked Control Systems With Unknown Dynamics and Dropout , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[24]  Rastko R. Selmic,et al.  Neural network control of a class of nonlinear systems with actuator saturation , 2006, IEEE Transactions on Neural Networks.

[25]  Yang Li,et al.  Adaptive Neural Network Control of AUVs With Control Input Nonlinearities Using Reinforcement Learning , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[26]  Rong Chen,et al.  Design and Verification of a Rail-Borne Energy Harvester for Powering Wireless Sensor Networks in the Railway Industry , 2017, IEEE Transactions on Intelligent Transportation Systems.

[27]  Guang-Ren Duan,et al.  $L_{\infty}$ and $L_{2}$ Low-Gain Feedback: Their Properties, Characterizations and Applications in Constrained Control , 2011, IEEE Transactions on Automatic Control.

[28]  Ali Saberi,et al.  Perfect regulation of linear discrete-time systems: A low-gain-based design approach , 1996, Autom..

[29]  James Lam,et al.  Semi-global stabilization of linear time-delay systems with control energy constraint , 2012, Autom..

[30]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[31]  Youmin Zhang,et al.  Adaptive Sliding Mode Fault Tolerant Attitude Tracking Control for Flexible Spacecraft Under Actuator Saturation , 2012, IEEE Transactions on Control Systems Technology.