Stochastic Optimal Control via Bellman’s Principle

This paper presents a strategy for finding optimal controls of non-linear systems subject to random excitations. The method is capable to generate global control solutions when state and control constraints are present. The solution is global in the sense that controls for all initial conditions in a region of the state space are obtained. The approach is based on Bellman's principle of optimality, the cumulant neglect closure method and the short-time Gaussian approximation. Problems with state-dependent diffusion terms, non-closeable hierarchies of moment equations for the states and singular state boundary condition are considered in the examples. The uncontrolled and controlled system responses are evaluated by creating a Markov chain with a control dependent transition probability matrix via the generalized cell mapping method. In all numerical examples, excellent controlled performances were obtained.

[1]  Alexander S. Bratus,et al.  Optimal Bounded Response Control for a Second-Order System Under a White-Noise Excitation , 2000 .

[2]  W. G. Wolfer,et al.  Numerical evaluation of path-integral solutions to Fokker-Planck equations. III. Time and functionally dependent coefficients. , 1987, Physical review. A, General physics.

[3]  C. Hsu,et al.  Cell-To-Cell Mapping A Method of Global Analysis for Nonlinear Systems , 1987 .

[4]  T. T. Soong,et al.  An Optimal Nonlinear Feedback Control Strategy for Randomly Excited Structural Systems , 2001 .

[5]  C. S. Hsu,et al.  Cumulant-neglect closure method for asymmetric non-linear systems driven by Gaussian white noise , 1989 .

[6]  Jian-Qiao Sun,et al.  Solution of Fixed Final State Optimal Control Problems via Simple Cell Mapping , 2000 .

[7]  Jian-Qiao Sun,et al.  Stochastic Optimal Control of Nonlinear Systems via Short-Time Gaussian Approximation and Cell Mapping , 2002 .

[8]  C. Hsu.  A discrete method of optimal control based upon the cell state space concept , 1985 .

[9]  Jian-Qiao Sun,et al.  Random vibration analysis of a non-linear system with dry friction damping by the short-time gaussian cell mapping method , 1995 .

[10]  J. Elgin.  The Fokker-Planck Equation: Methods of Solution and Applications , 1984 .

[11]  A. Naess,et al.  Efficient path integration methods for nonlinear dynamic systems , 2000 .

[12]  Alexander S. Bratus,et al.  Optimal bounded control of steady-state random vibrations , 2000 .

[13]  T. Aldemir,et al.  CONTINUOUS CELL-TO-CELL MAPPING , 1997 .

[14]  H. Kushner.  Numerical Methods for Stochastic Control Problems in Continuous Time , 2000 .

[15]  C. Hsu,et al.  The Generalized Cell Mapping Method in Nonlinear Random Vibration Based Upon Short-Time Gaussian Approximation , 1990 .

[16]  X. Zhou,et al.  Stochastic Controls: Hamiltonian Systems and HJB Equations , 1999 .

[17]  Yu-Kweng Michael Lin,et al.  Probabilistic Structural Dynamics: Advanced Theory and Applications , 1967 .