Probabilistic DHP adaptive critic for nonlinear stochastic control systems

Following the recently developed algorithms for fully probabilistic control design for general dynamic stochastic systems (Herzallah & Káarnáy, 2011; Kárný, 1996), this paper presents the solution to the probabilistic dual heuristic programming (DHP) adaptive critic method (Herzallah & Káarnáy, 2011) and randomized control algorithm for stochastic nonlinear dynamical systems. The purpose of the randomized control input design is to make the joint probability density function of the closed loop system as close as possible to a predetermined ideal joint probability density function. This paper completes the previous work (Herzallah & Káarnáy, 2011; Kárný, 1996) by formulating and solving the fully probabilistic control design problem on the more general case of nonlinear stochastic discrete time systems. A simulated example is used to demonstrate the use of the algorithm and encouraging results have been obtained.

[1]  V. Solo.  Stochastic adaptive control and Martingale limit theory , 1990 .

[2]  Pieter Eykhoff,et al.  Trends and progress in system identification , 1981 .

[3]  Y. Ho,et al.  A Bayesian approach to problems in stochastic estimation and control , 1964 .

[4]  Hong Wang,et al.  Robust control of the output probability density functions for multivariable stochastic systems with guaranteed stability , 1999, IEEE Trans. Autom. Control..

[5]  Xingyu Wang,et al.  Robust $H_{\infty}$ Control for Nonlinear Stochastic Systems: A Sliding-Mode Approach , 2008, IEEE Transactions on Automatic Control.

[6]  A. Barto,et al.  ModelBased Adaptive Critic Designs , 2004 .

[7]  Sean P. Meyn,et al.  A NEW APPROACH TO STOCHASTIC .4DAPTIVE CONTROL , 1986 .

[8]  K. Åström.  Introduction to Stochastic Control Theory , 1970 .

[9]  S. N. Balakrishnan,et al.  State-constrained agile missile control with adaptive-critic-based neural networks , 2002, IEEE Trans. Control. Syst. Technol..

[10]  Hong Wang,et al.  Robust control of the output probability density functions for multivariable stochastic systems , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[11]  Paul J. Werbos,et al.  Approximate dynamic programming for real-time control and neural modeling , 1992 .

[12]  F. Graybill,et al.  Matrices with Applications in Statistics. , 1984 .

[13]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[14]  Hong Wang,et al.  ILC-Based Fixed-Structure Controller Design for Output PDF Shaping in Stochastic Systems Using LMI Techniques , 2009, IEEE Transactions on Automatic Control.

[15]  Masahiro Ono,et al.  A Probabilistic Particle-Control Approximation of Chance-Constrained Stochastic Predictive Control , 2010, IEEE Transactions on Robotics.

[16]  R. Herzallah.  Probabilistic Control for Uncertain Systems , 2012, 1801.02074.

[17]  Guan Xin-ping.  Robust H_∞ Control for Nonlinear Systems , 2002 .

[18]  Randa Herzallah,et al.  A Bayesian Perspective on Stochastic Neurocontrol , 2008, IEEE Transactions on Neural Networks.

[19]  George G. Lendaris,et al.  Proposed framework for applying adaptive critics in real-time realm , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[20]  Brian D. O. Anderson,et al.  Linear Optimal Control , 1971 .

[21]  F. Graybill,et al.  Matrices with Applications in Statistics. , 1984 .

[22]  R. Bellman,et al.  Dynamic Programming and Markov Processes , 1960 .

[23]  Nilesh V. Kulkarni,et al.  Intelligent engine control using an adaptive critic , 2003, IEEE Trans. Control. Syst. Technol..

[24]  S. N. Balakrishnan,et al.  Adaptive-critic based neural networks for aircraft optimal control , 1996 .

[25]  Václav Peterka,et al.  Bayesian system identification , 1979, Autom..

[26]  Jianhua Zhang,et al.  Bounded stochastic distributions control for pseudo-ARMAX stochastic systems , 2001, IEEE Trans. Autom. Control..

[27]  Hong Wang,et al.  Minimum entropy control of non-Gaussian dynamic stochastic systems , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[28]  H.A.P. Blom,et al.  Embedding adaptive JLQG into LQ martingale control with a completely observable stochastic control matrix , 1996, IEEE Trans. Autom. Control..

[29]  Miroslav Kárný,et al.  Fully probabilistic control design in an adaptive critic framework , 2011, Neural Networks.

[30]  Visakan Kadirkamanathan,et al.  Functional Adaptive Control: An Intelligent Systems Approach , 2012 .

[31]  Randa Herzallah.  Adaptive critic methods for stochastic systems with input-dependent noise , 2007, Autom..

[32]  Miroslav Kárný.  Towards Fully Probabilistic Control Design , 1995 .

[33]  Chuan-Kai Lin.  Radial basis function neural network-based adaptive critic control of induction motors , 2011, Appl. Soft Comput..

[34]  Tatiana V. Guy,et al.  Fully probabilistic control design , 2006, Syst. Control. Lett..

[35]  N. M. Filatov,et al.  Adaptive predictive control policy for nonlinear stochastic systems , 1995, IEEE Trans. Autom. Control..

[36]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[37]  Miroslav Kárný,et al.  Towards fully probabilistic control design , 1996, Autom..