Boundary Control of Linear Uncertain 1-D Parabolic PDE Using Approximate Dynamic Programming

This paper develops a near optimal boundary control method for distributed parameter systems governed by uncertain linear 1-D parabolic partial differential equations (PDE) by using approximate dynamic programming. A quadratic surface integral is proposed to express the optimal cost functional for the infinite-dimensional state space. Accordingly, the Hamilton–Jacobi–Bellman (HJB) equation is formulated in the infinite-dimensional domain without using any model reduction. Subsequently, a neural network identifier is developed to estimate the unknown spatially varying coefficient in PDE dynamics. Novel tuning law is proposed to guarantee the boundedness of identifier approximation error in the PDE domain. A radial basis network (RBN) is subsequently proposed to generate an approximate solution for the optimal surface kernel function online. The tuning law for near optimal RBN weights is created, such that the HJB equation error is minimized while the dynamics are identified and closed-loop system remains stable. Ultimate boundedness (UB) of the closed-loop system is verified by using the Lyapunov theory. The performance of the proposed controller is successfully confirmed by simulation on an unstable diffusion–reaction process.

[1]  Kurt Hornik,et al.  Degree of Approximation Results for Feedforward Networks Approximating Unknown Mappings and Their Derivatives , 1994, Neural Computation.

[2]  Hao Xu,et al.  Neural network dynamic progamming constrained control of distributed parameter systems governed by parabolic partial differential equations with application to diffusion-reaction processes , 2014, 2014 IEEE International Symposium on Intelligent Control (ISIC).

[3]  Randal W. Beard,et al.  Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation , 1997, Autom..

[4]  Frank L. Lewis,et al.  Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2010, Autom..

[5]  Angelo Alessandri,et al.  Feedback Optimal Control of Distributed Parameter Systems by Using Finite-Dimensional Approximation Schemes , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[6]  Vanessa Hertzog,et al.  Computational Fluid Mechanics And Heat Transfer , 2016 .

[7]  P. Lions,et al.  Hamilton-Jacobi equations in infinite dimensions I. Uniqueness of viscosity solutions , 1985 .

[8]  Han-Xiong Li,et al.  Adaptive Optimal Control of Highly Dissipative Nonlinear Spatially Distributed Processes With Neuro-Dynamic Programming , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[9]  Qiuye Sun,et al.  Adaptive critic design-based robust neural network control for nonlinear distributed parameter systems with unknown dynamics , 2015, Neurocomputing.

[10]  Miroslav Krstic,et al.  Finite-time multi-agent deployment: A nonlinear PDE motion planning approach , 2011, Autom..

[11]  Han-Xiong Li,et al.  Adaptive Neural Control Design for Nonlinear Distributed Parameter Systems With Persistent Bounded Disturbances , 2009, IEEE Transactions on Neural Networks.

[12]  Han-Xiong Li,et al.  Distributed Fuzzy Control Design of Nonlinear Hyperbolic PDE Systems With Application to Nonisothermal Plug-Flow Reactor , 2011, IEEE Transactions on Fuzzy Systems.

[13]  Sarangapani Jagannathan,et al.  Fault diagnosis of a class of distributed parameter systems modeled by parabolic partial differential equations , 2014, 2014 American Control Conference.

[14]  Frank L. Lewis,et al.  Adaptive optimal control for continuous-time linear systems based on policy iteration , 2009, Autom..

[15]  Martin D. Buhmann,et al.  Radial Basis Functions: Theory and Implementations: Preface , 2003 .

[16]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[17]  Sarangapani Jagannathan,et al.  Online Optimal Control of Affine Nonlinear Discrete-Time Systems With Unknown Internal Dynamics by Using Time-Based Policy Update , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[19]  R. Triggiani,et al.  Control Theory for Partial Differential Equations: Continuous and Approximation Theories , 2000 .

[20]  J. Lions.  Optimal Control of Systems Governed by Partial Differential Equations , 1971 .

[21]  Hao Xu,et al.  Near optimal boundary control of distributed parameter systems modeled as parabolic pdes by using finite difference neural network approximation , 2014, 53rd IEEE Conference on Decision and Control.

[22]  W. Rudin.  Principles of mathematical analysis , 1964 .

[23]  A. Quarteroni.  Numerical Models for Differential Problems , 2009 .

[24]  Frank L. Lewis,et al.  Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem , 2009, 2009 International Joint Conference on Neural Networks.

[25]  E. Kreyszig.  Introductory Functional Analysis With Applications , 1978 .

[26]  Anthony W. Leung,et al.  Systems of Nonlinear Partial Differential Equations: Applications to Biology and Engineering , 1989 .

[27]  Miroslav Krstic,et al.  Output-feedback adaptive control of a wave PDE with boundary anti-damping , 2014, Autom..

[28]  Donald R. Smith.  Variational methods in optimization , 1974 .

[29]  Huaguang Zhang,et al.  Online optimal tracking control of continuous-time linear systems with unknown dynamics by using adaptive dynamic programming , 2014, Int. J. Control.

[30]  F. Lewis,et al.  Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers , 2012, IEEE Control Systems.

[31]  R. Pletcher,et al.  Computational Fluid Mechanics and Heat Transfer. By D. A ANDERSON, J. C. TANNEHILL and R. H. PLETCHER. Hemisphere, 1984. 599 pp. $39.95. , 1986, Journal of Fluid Mechanics.

[32]  M. Krstić.  Boundary Control of PDEs: A Course on Backstepping Designs , 2008 .

[33]  J. Raymond,et al.  Hamilton-Jacobi equations for control problems of parabolic equations , 2006 .

[34]  P. Christofides,et al.  Nonlinear and Robust Control of PDE Systems: Methods and Applications to Transport-Reaction Processes , 2002 .

[35]  Piermarco Cannarsa,et al.  Infinite-Dimensional Hamilton--Jacobi Equations and Dirichlet Boundary Control Problems of Parabolic Type , 1996 .

[36]  J. Weidmann.  Linear Operators in Hilbert Spaces , 1980 .

[37]  Miroslav Krstic,et al.  PDE Boundary Control for Flexible Articulated Wings on a Robotic Aircraft , 2013, IEEE Transactions on Robotics.

[38]  Martin Berzins,et al.  A Method for the Spatial Discretization of Parabolic Equations in One Space Variable , 1990, SIAM J. Sci. Comput..

[39]  Frank L. Lewis,et al.  Multiple Actor-Critic Structures for Continuous-Time Optimal Control Using Input-Output Data , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Boris Vexler,et al.  Adaptive Finite Elements for Elliptic Optimization Problems with Control Constraints , 2008, SIAM J. Control. Optim..