Algorithm and stability of ATC receding horizon control

Receding horizon control (RHC), also known as model predictive control (MPC), is a suboptimal control scheme that solves a finite horizon open-loop optimal control problem in an infinite horizon context and yields a measured state feedback control law. A lot of efforts have been made to study the closed-loop stability, leading to various stability conditions involving constraints on either the terminal state, or the terminal cost, or the horizon size, or their different combinations. In this paper, we propose a modified RHC scheme, called adaptive terminal cost RHC (ATC-RHC). The control law generated by ATC-RHC algorithm converges to the solution of the infinite horizon optimal control problem. Moreover, it ensures the closed-loop system to be uniformly ultimately exponentially stable without imposing any constraints on the terminal state, the horizon size, or the terminal cost. Finally we show that when the horizon size is one, the underlying problems of ATC-RHC and heuristic dynamic programming (HDP) are the same. Thus, ATC-RHC can be implemented using HDP techniques without knowing the system matrix A.

[1]  Thomas Parisini,et al.  A receding-horizon regulator for nonlinear systems and a neural approximation , 1995, Autom..

[2]  R. Godson.  Elements of intelligence , 1979 .

[3]  S. Joe Qin,et al.  A survey of industrial model predictive control technology , 2003 .

[4]  David Q. Mayne,et al.  Constrained model predictive control: Stability and optimality , 2000, Autom..

[5]  F. Lewis,et al.  Continuous-Time ADP for Linear Systems with Partially Unknown Dynamics , 2007, 2007 IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning.

[6]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[7]  V. Wertz,et al.  Adaptive Optimal Control: The Thinking Man's G.P.C. , 1991 .

[8]  C. V. Rao,et al.  Steady states and constraints in model predictive control , 1999 .

[9]  W. Rugh.  Linear System Theory , 1992 .

[10]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Wook Hyun Kwon,et al.  Stability-Guaranteed Horizon Size for Receding Horizon Control , 2007, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[12]  E. Gilbert,et al.  Optimal infinite-horizon feedback laws for a general class of constrained discrete-time systems: Stability and moving-horizon approximations , 1988 .

[13]  L. Magni,et al.  Stability margins of nonlinear receding-horizon control via inverse optimality , 1997 .

[14]  W. Kwon,et al.  A modified quadratic cost problem and feedback stabilization of a linear system , 1977 .

[15]  Jie Yu,et al.  Unconstrained receding-horizon control of nonlinear systems , 2001, IEEE Trans. Autom. Control..

[16]  Robert R. Bitmead,et al.  Riccati Difference and Differential Equations: Convergence, Monotonicity and Stability , 1991 .

[17]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[18]  Frank L. Lewis,et al.  Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ Control , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  N. L. Ricker,et al.  Adaptive optimal control: The thinking man's GPC : R. R. Bitmead, M. Gevers and V. Wertz , 1993, Autom..

[20]  F. Allgöwer,et al.  A quasi-infinite horizon nonlinear model predictive control scheme with guaranteed stability , 1997 .

[21]  James A. Primbs,et al.  Feasibility and stability of constrained finite receding horizon control , 2000, Autom..

[22]  Warren B. Powell,et al.  Handbook of Learning and Approximate Dynamic Programming , 2006, IEEE Transactions on Automatic Control.

[23]  Chi-Tsong Chen,et al.  Linear System Theory and Design , 1995 .

[24]  Jay H. Lee,et al.  Model predictive control: past, present and future , 1999 .

[25]  Tomas Landelius,et al.  Reinforcement Learning and Distributed Local Model Synthesis , 1997 .

[26]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[27]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.

[28]  Luigi Chisci,et al.  Dual-Receding Horizon Control of Constrained Discrete Time Systems , 1996, Eur. J. Control.

[29]  Paul J. Werbos,et al.  Building and Understanding Adaptive Systems: A Statistical/Numerical Approach to Factory Automation and Brain Research , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[30]  D. Q. Mayne,et al.  Suboptimal model predictive control (feasibility implies stability) , 1999, IEEE Trans. Autom. Control..

[31]  R. Bellman,et al.  Dynamic Programming and Markov Processes , 1960 .

[32]  Paul J. Webros.  A menu of designs for reinforcement learning over time , 1990 .

[33]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[34]  D. Mayne,et al.  Robust receding horizon control of constrained nonlinear systems , 1993, IEEE Trans. Autom. Control..

[35]  Chris Watkins,et al.  Learning from delayed rewards , 1989 .

[36]  W. Kwon,et al.  On feedback stabilization of time-varying discrete linear systems , 1978 .

[37]  G. Nicolao,et al.  Stabilizing receding-horizon control of nonlinear time-varying systems , 1998, IEEE Trans. Autom. Control..

[38]  John Hauser,et al.  On the stability of receding horizon control with a general terminal cost , 2005, IEEE Transactions on Automatic Control.

[39]  Andrew R. Teel,et al.  Model predictive control: for want of a local control Lyapunov function, all is not lost , 2005, IEEE Transactions on Automatic Control.

[40]  Wook Hyun Kwon,et al.  On stability of constrained receding horizon control with finite terminal weighting matrix , 1997, 1997 European Control Conference (ECC).