Approximate Optimal Control Design for Nonlinear One-Dimensional Parabolic PDE Systems Using Empirical Eigenfunctions and Neural Network

This paper addresses the approximate optimal control problem for a class of parabolic partial differential equation (PDE) systems with nonlinear spatial differential operators. An approximate optimal control design method is proposed on the basis of the empirical eigenfunctions (EEFs) and neural network (NN). First, based on the data collected from the PDE system, the Karhunen-Loève decomposition is used to compute the EEFs. With those EEFs, the PDE system is formulated as a high-order ordinary differential equation (ODE) system. To further reduce its dimension, the singular perturbation (SP) technique is employed to derive a reduced-order model (ROM), which can accurately describe the dominant dynamics of the PDE system. Second, the Hamilton-Jacobi-Bellman (HJB) method is applied to synthesize an optimal controller based on the ROM, where the closed-loop asymptotic stability of the high-order ODE system can be guaranteed by the SP theory. By dividing the optimal control law into two parts, the linear part is obtained by solving an algebraic Riccati equation, and a new type of HJB-like equation is derived for designing the nonlinear part. Third, a control update strategy based on successive approximation is proposed to solve the HJB-like equation, and its convergence is proved. Furthermore, an NN approach is used to approximate the cost function. Finally, we apply the developed approximate optimal control method to a diffusion-reaction process with a nonlinear spatial operator, and the simulation results illustrate its effectiveness.

[1]  Frank L. Lewis,et al.  Nearly optimal state feedback control of constrained nonlinear systems using a neural networks HJB approach , 2004, Annu. Rev. Control..

[2]  Panagiotis D. Christofides,et al.  Output feedback control of parabolic PDE systems with nonlinear spatial differential operators , 1999 .

[3]  Hans Zwart,et al.  An Introduction to Infinite-Dimensional Linear Systems Theory , 1995, Texts in Applied Mathematics.

[4]  Frank L. Lewis,et al.  Discrete-Time Nonlinear HJB Solution Using Approximate Dynamic Programming: Convergence Proof , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[5]  Frank L. Lewis,et al.  Adaptive Critic Designs for Discrete-Time Zero-Sum Games With Application to $H_{\infty}$ Control , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[7]  F. Tung,et al.  Optimum Control of Distributed-Parameter Systems , 1964 .

[8]  Ibrahim Sadek,et al.  Optimal control of a parabolic distributed parameter system via orthogonal polynomials , 1998 .

[9]  S. Ravindran.  A reduced-order approach for optimal control of fluids using proper orthogonal decomposition , 2000 .

[10]  Huaguang Zhang,et al.  A Novel Infinite-Time Optimal Tracking Control Scheme for a Class of Discrete-Time Nonlinear Systems via the Greedy HDP Iteration Algorithm , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[11]  Hector O. Fattorini,et al.  Infinite Dimensional Optimization and Control Theory: References , 1999 .

[12]  P. Christofides,et al.  Dynamic optimization of dissipative PDE systems using nonlinear order reduction , 2002 .

[13]  R. Rogers,et al.  An introduction to partial differential equations , 1993 .

[14]  Frank L. Lewis,et al.  Guest Editorial: Special Issue on Adaptive Dynamic Programming and Reinforcement Learning in Feedback Control , 2008, IEEE Trans. Syst. Man Cybern. Part B.

[15]  Randal W. Beard,et al.  Galerkin approximations of the generalized Hamilton-Jacobi-Bellman equation , 1997, Autom..

[16]  Panagiotis D. Christofides,et al.  Optimal control of diffusion-convection-reaction processes using reduced-order models , 2008, Comput. Chem. Eng..

[17]  A. G. Butkovskiĭ,et al.  Distributed control systems , 1969 .

[18]  Andrew J. Newman.  Model Reduction via the Karhunen-Loeve Expansion Part II: Some Elementary Examples , 1996 .

[19]  Huaguang Zhang,et al.  Neural-Network-Based Near-Optimal Control for a Class of Discrete-Time Affine Nonlinear Systems With Control Constraints , 2009, IEEE Transactions on Neural Networks.

[20]  Frank L. Lewis,et al.  Reinforcement Learning for Partially Observable Dynamic Processes: Adaptive Dynamic Programming Using Measured Output Data , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Eugenio Schuster,et al.  Sequential linear quadratic control of bilinear parabolic PDEs based on POD model reduction , 2011, Autom..

[22]  George N. Saridis,et al.  An Approximation Theory of Optimal Control for Trainable Manipulators , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[23]  Frank L. Lewis,et al.  A Neural Network Solution for Fixed-Final Time Optimal Control of Nonlinear Systems , 2006, 2006 14th Mediterranean Conference on Control and Automation.

[24]  Jiongmin Yong,et al.  Optimal Control Theory for Infinite Dimensional Systems , 1994 .

[25]  D. Zheng,et al.  System identification and model-based control for a class of distributed parameter systems , 2003 .

[26]  Han-Xiong Li,et al.  Spatio-Temporal Modeling of Nonlinear Distributed Parameter Systems: A Time/Space Separation Based Approach , 2011 .

[27]  G. Saridis,et al.  Approximate Solutions to the Time-Invariant Hamilton–Jacobi–Bellman Equation , 1998 .

[28]  P. Holmes,et al.  Turbulence, Coherent Structures, Dynamical Systems and Symmetry , 1996 .

[29]  Joseph J. Winkin,et al.  LQ control design of a class of hyperbolic PDE systems: Application to fixed-bed reactor , 2009, Autom..

[30]  Radhakant Padhi,et al.  Adaptive-critic based optimal neuro control synthesis for distributed parameter systems , 2001, Autom..

[31]  Dimitri P. Bertsekas,et al.  Dynamic programming and optimal control, 3rd Edition , 2005 .

[32]  W. Ray,et al.  Identification and control of distributed parameter systems by means of the singular value decomposition , 1995 .

[33]  P. Daoutidis,et al.  Finite-dimensional control of parabolic PDE systems using approximate inertial manifolds , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[34]  Bernard Widrow,et al.  Punish/Reward: Learning with a Critic in Adaptive Threshold Systems , 1973, IEEE Trans. Syst. Man Cybern..

[35]  P. Christofides,et al.  Finite-dimensional approximation and control of non-linear parabolic PDE systems , 2000 .

[36]  Costas J. Spanos,et al.  Advanced process control , 1989 .

[37]  Frank L. Lewis,et al.  Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach , 2005, Autom..

[38]  Radhakant Padhi,et al.  Proper orthogonal decomposition based optimal neurocontrol synthesis of a chemical reactor process using approximate dynamic programming , 2003, Neural Networks.

[39]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[40]  Panagiotis D. Christofides,et al.  An Input/Output Approach to the Optimal Transition Control of a Class of Distributed Chemical Reactors , 2007, ACC.

[41]  Denis Dochain,et al.  Optimal LQ-Feedback Regulation of a Nonisothermal Plug Flow Reactor Model by Spectral Factorization , 2007, IEEE Transactions on Automatic Control.

[42]  Frank L. Lewis,et al.  Fixed-Final Time Constrained Optimal Control of Nonlinear Systems Using Neural Network HJB Approach , 2006, CDC.

[43]  Leonidas G. Bleris,et al.  Low-order empirical modeling of distributed parameter systems using temporal and spatial eigenfunctions , 2005, Comput. Chem. Eng..

[44]  Han-Xiong Li,et al.  Spatio-Temporal Modeling of Nonlinear Distributed Parameter Systems , 2011 .