Improving the performance of globalized dual heuristic programming for fault tolerant control through an online learning supervisor

An advanced reconfigurable controller enhanced by a multiple model architecture is proposed as a tool to achieve fault tolerance in complex nonlinear systems. The most complete adaptive critic design, globalized dual heuristic programming (GDHP), constitutes a highly flexible nonlinear adaptive controller responsible for the generation of new control solutions for novel plant dynamics introduced by unknown faults. The main contribution of the presented work focuses on a novel fault tolerant control supervisor. Working on a higher hierarchical level, the proposed supervisor makes use of two quality indices to perform fault detection, identification and isolation based on the knowledge stored in a dynamic model bank (DMB). In the event of abrupt known faults, such knowledge is then used to greatly reduce the reconfiguration time of the GDHP controller. The synergy of the proposed supervisor and GDHP goes beyond, as solutions designed by the controller to previously unknown faults are autonomously added to the model bank. The fine interrelations between the algorithm's subsystems and its advanced capabilities are illustrated through extensive numerical simulations of a single-input single-output (SISO) linear system and of a multiple-input multiple-output (MIMO) nonlinear system, both subject to a series of fault scenarios involving expected and unexpected, abrupt and incipient faults. Note to Practitioners-The raising complexity of physical plants and control missions inevitably leads to increasing occurrence, diversity and severity of faults. Take automated production process as an example, the extent of time a plant is capable of maintaining acceptable performance levels is now considered to be the single factor with the highest impact on profitability. For a growing number of plants, it has become impractical to list all possible fault scenarios in order to take the necessary steps to assure continuous healthy operation. Therefore, it is now essential to have a control algorithm dedicated to the provision of tailored control solutions capable of maintaining stability and as much performance as possible during the occurrence of faults. This is the goal of fault tolerant control. In the presented paper, the most complete adaptive critic design, globalized dual heuristic programming (GDHP), is responsible for the generation of new control solutions for novel plant dynamics introduced by faults unknown at design time. A highly flexible nonlinear adaptive controller, GDHP is capable of dealing with both abrupt and incipient (gradually changing dynamics) faults. Working on a higher hierarchical level, a novel fault supervisor is introduced which makes use of two quality indices to perform fault detection, identification and isolation based on the knowledge stored in a dynamic model bank (DMB). In the event of abrupt known faults, such knowledge is then used to greatly reduce the convergence time of the GDHP controller. The synergy of the proposed supervisor and GDHP goes further, as solutions designed by the controller to previously unknown faults are autonomously added to the model bank in fact allowing the supervisor to learn new fault scenarios and their solutions as they occur. The fine interrelations between the algorithm's subsystems and its advanced capabilities are illustrated through extensive numerical simulations.

[1]  Karl Johan Åström,et al.  Control of complex systems , 2001 .

[2]  Yixin Diao,et al.  Intelligent fault-tolerant control using adaptive and learning methods , 2002 .

[3]  George G. Lendaris,et al.  Globally convergent approximate dynamic programming applied to an autolander , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[4]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[5]  Youmin Zhang,et al.  Integrated active fault-tolerant control using IMM approach , 2001 .

[6]  Edgar N. Sánchez,et al.  Adaptive recurrent neural control for nonlinear system tracking , 2000, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Paul J. Werbos.  New designs for universal stability in classical adaptive control and reinforcement learning , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[8]  George G. Lendaris,et al.  More on training strategies for critic and action neural networks in dual heuristic programming method , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[9]  Jesus M. de la Cruz,et al.  An Eigenstructure Assignment approach (2) , 1997 .

[10]  Mogens Blanke,et al.  Fault-tolerant control systems — A holistic view , 1997 .

[11]  Yixin Diao,et al.  Fault tolerant system design using adaptive estimation and control , 2000 .

[12]  Phayung Meesad,et al.  An effective neuro-fuzzy paradigm for machinery condition health monitoring , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[13]  S. J. Qin,et al.  Detection and identification of faulty sensors with maximized sensitivity , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[14]  H. E. Rauch,et al.  Autonomous control reconfiguration , 1995 .

[15]  S. A. Bøgh,et al.  Fault Tolerant Control Systems , 1997 .

[16]  Youmin Zhang,et al.  Analysis of the stochastic stability for fault tolerant control systems , 1999, Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304).

[17]  G G Yen,et al.  Winner take all experts network for sensor validation. , 2001, ISA transactions.

[18]  Mogens Blanke,et al.  FAULT MONITORING AND RE-CONFIGURABLE CONTROL FOR A SHIP PROPULSION PLANT , 1998 .

[19]  Sai-Ming Li,et al.  On-line failure detection and identification (FDI) and adaptive reconfigurable control (ARC) in aerospace applications , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[20]  Ron J. Patton,et al.  Fault-Tolerant Control: The 1997 Situation , 1997 .

[21]  George G. Lendaris,et al.  Controller design (from scratch) using approximate dynamic programming , 2000, Proceedings of the 2000 IEEE International Symposium on Intelligent Control. Held jointly with the 8th IEEE Mediterranean Conference on Control and Automation (Cat. No.00CH37147).

[22]  Roberto A. Santiago,et al.  Adaptive critic designs: A case study for neurocontrol , 1995, Neural Networks.

[23]  Panos J. Antsaklis,et al.  Actuator and sensor failure accommodation with incomplete information , 1992, [Proceedings 1992] The First IEEE Conference on Control Applications.

[24]  N. E. Wu,et al.  Concepts and methods in fault-tolerant control , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[25]  Sean R Eddy,et al.  What is dynamic programming? , 2004, Nature Biotechnology.

[26]  George G. Lendaris,et al.  Training strategies for critic and action neural networks in dual heuristic programming method , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[27]  Snehasis Mukhopadhyay,et al.  Adaptive control using neural networks and approximate models , 1997, IEEE Trans. Neural Networks.

[28]  Dimitar Petrov Filev,et al.  Intelligent adaptive control using multiple models , 2001, Proceeding of the 2001 IEEE International Symposium on Intelligent Control (ISIC '01) (Cat. No.01CH37206).

[29]  Gary G. Yen,et al.  Online multiple-model-based fault diagnosis and accommodation , 2003, IEEE Trans. Ind. Electron..

[30]  Kumpati S. Narendra,et al.  Adaptation and learning using multiple models, switching, and tuning , 1995 .

[31]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.