Advanced policy learning near-optimal regulation

Designing advanced design techniques for feedback stabilization and optimization of complex systems is important to the modern control field. In this paper, a near-optimal regulation method for general nonaffine dynamics is developed with the help of policy learning. For addressing the nonaffine nonlinearity, a pre-compensator is constructed, so that the augmented system can be formulated as affine-like form. Different cost functions are defined for original and transformed controlled plants and then their relationship is analyzed in detail. Additionally, an adaptive critic algorithm involving stability guarantee is employed to solve the augmented optimal control problem. At last, several case studies are conducted for verifying the stability, robustness, and optimality of a torsional pendulum plant with suitable cost.

[1]  Yang Xiong,et al.  Adaptive Dynamic Programming with Applications in Optimal Control , 2017 .

[2]  Xin Zhang,et al.  Data-Driven Robust Approximate Optimal Tracking Control for Unknown General Nonlinear Systems Using Adaptive Dynamic Programming Method , 2011, IEEE Transactions on Neural Networks.

[3]  Haibo He,et al.  Adaptive Critic Nonlinear Robust Control: A Survey , 2017, IEEE Transactions on Cybernetics.

[4]  Haibo He,et al.  Model-Free Adaptive Control for Unknown Nonlinear Zero-Sum Differential Game , 2018, IEEE Transactions on Cybernetics.

[5]  Frank L. Lewis,et al.  Special Issue on Deep Reinforcement Learning and Adaptive Dynamic Programming , 2018, IEEE Trans. Multimedia.

[6]  Jing Na,et al.  Observer-based adaptive optimal control for unknown singularly perturbed nonlinear systems with input constraints , 2017, IEEE/CAA Journal of Automatica Sinica.

[7]  Xiong Yang,et al.  Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints , 2014, Int. J. Control.

[8]  Richard Saeks,et al.  Adaptive critic control and functional link neural networks , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[9]  Zhong-Ping Jiang,et al.  Adaptive dynamic programming and optimal control of nonlinear nonaffine systems , 2014, Autom..

[10]  Frank L. Lewis,et al.  H∞ Control of Nonaffine Aerial Systems Using Off-policy Reinforcement Learning , 2016, Unmanned Syst..

[11]  Guanghong Yang,et al.  Adaptive nearly optimal control for a class of continuous-time nonaffine nonlinear systems with inequality constraints. , 2017, ISA transactions.

[12]  Haibo He,et al.  Intelligent Critic Control With Disturbance Attenuation for Affine Dynamics Including an Application to a Microgrid System , 2017, IEEE Transactions on Industrial Electronics.

[13]  Derong Liu,et al.  Residential energy scheduling for variable weather solar energy based on adaptive dynamic programming , 2018, IEEE/CAA Journal of Automatica Sinica.

[14]  Zhong-Ping Jiang,et al.  Learning-Based Adaptive Optimal Tracking Control of Strict-Feedback Nonlinear Systems , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[15]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[16]  Frank L. Lewis,et al.  Off-Policy Q-Learning: Set-Point Design for Optimizing Dual-Rate Rougher Flotation Operational Processes , 2018, IEEE Transactions on Industrial Electronics.

[17]  Haibo He,et al.  Intelligent Optimal Control With Critic Learning for a Nonlinear Overhead Crane System , 2018, IEEE Transactions on Industrial Informatics.