Direct Adaptive Optimal Control for Uncertain Continuous-Time LTI Systems Without Persistence of Excitation

This brief presents a novel direct adaptive optimal controller design for uncertain continuous-time linear time-invariant systems. The optimal gain parameter, obtained from the Riccati equation, is continuously estimated without using knowledge of the system dynamics, rather information rich past, and current data along the system trajectory is used for parameter estimation. This approach guarantees parameter convergence to the close neighborhood of the optimal controller by relaxing the restrictive persistence of excitation condition, typically required for achieving parameter convergence in approximate/adaptive optimal control methods. A Lyapunov-based analysis establishes the uniformly ultimately bounded stability of the designed controller. Further a simulation example demonstrates the efficacy of the proposed result.

[1]  Jin Bae Park,et al.  Generalized predictive control based on self-recurrent wavelet neural network for stable path tracking of mobile robots: adaptive learning rates approach , 2006, IEEE Transactions on Circuits and Systems I: Regular Papers.

[2]  Kyriakos G. Vamvoudakis,et al.  Asymptotically Stable Adaptive–Optimal Control Algorithm With Saturating Actuators and Relaxed Persistence of Excitation , 2016, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Girish Chowdhary,et al.  Concurrent learning adaptive control of linear systems with exponentially convergent bounds , 2013 .

[4]  Zhong-Ping Jiang,et al.  Robust Adaptive Dynamic Programming for Large-Scale Systems With an Application to Multimachine Power Systems , 2012, IEEE Transactions on Circuits and Systems II: Express Briefs.

[5]  Luigi Fortuna,et al.  Reinforcement Learning and Adaptive Dynamic Programming for Feedback Control , 2009 .

[6]  Frank L. Lewis,et al.  A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems , 2013, Autom..

[7]  B. Anderson,et al.  Optimal control: linear quadratic methods , 1990 .

[8]  Zhong-Ping Jiang,et al.  Computational adaptive optimal control for continuous-time linear systems with completely unknown dynamics , 2012, Autom..

[9]  Sumit Kumar Jha,et al.  Data-Driven Adaptive LQR for Completely Unknown LTI Systems , 2017 .

[10]  Sumit Kumar Jha,et al.  On-policy Q-learning for adaptive optimal control , 2014, 2014 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL).

[11]  Travis E. Gibson,et al.  Projection Operator in Adaptive Systems , 2011, ArXiv.

[12]  Andrew G. Barto,et al.  Adaptive linear quadratic control using policy iteration , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[13]  Huanshui Zhang,et al.  Optimal Control and Stabilization for Networked Control Systems With Packet Dropout and Input Delay , 2017, IEEE Transactions on Circuits and Systems II: Express Briefs.

[14]  Sean P. Meyn,et al.  Q-learning and Pontryagin's Minimum Principle , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[15]  Warren E. Dixon,et al.  Model-based reinforcement learning for approximate optimal regulation , 2016, Autom..

[16]  Zhen Chen,et al.  Near Optimal Control Based on the Tensor-Product Technique , 2017, IEEE Transactions on Circuits and Systems II: Express Briefs.

[17]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[18]  Dongbin Zhao,et al.  Model-Free Optimal Control for Affine Nonlinear Systems With Convergence Analysis , 2015, IEEE Transactions on Automation Science and Engineering.

[19]  Girish Chowdhary,et al.  Concurrent learning for convergence in adaptive control without persistency of excitation , 2010, 49th IEEE Conference on Decision and Control (CDC).

[20]  Jae Young Lee,et al.  Integral Q-learning and explorized policy iteration for adaptive optimal control of continuous-time linear systems , 2012, Autom..

[21]  Frank L. Lewis,et al.  Adaptive optimal control for continuous-time linear systems based on policy iteration , 2009, Autom..

[22]  Frank L. Lewis,et al.  Adaptive Optimal Control of Unknown Constrained-Input Systems Using Policy Iteration and Neural Networks , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[23]  Yim-Shu Lee,et al.  Design and analysis of an optimal controller for parallel multi-inverter systems , 2006, IEEE Transactions on Circuits and Systems II: Express Briefs.

[24]  Pinaki Mazumder,et al.  A Scalable Low-Power Reconfigurable Accelerator for Action-Dependent Heuristic Dynamic Programming , 2018, IEEE Transactions on Circuits and Systems I: Regular Papers.

[25]  Antonio Loría,et al.  Adaptive Observers With Persistency of Excitation for Synchronization of Chaotic Systems , 2009, IEEE Transactions on Circuits and Systems I: Regular Papers.

[26]  Derong Liu,et al.  Data-Based Adaptive Critic Designs for Nonlinear Robust Optimal Control With Uncertain Dynamics , 2016, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[27]  Kyriakos G. Vamvoudakis,et al.  Q-learning for continuous-time linear systems: A model-free infinite horizon optimal control approach , 2017, Syst. Control. Lett..