Model-free control of nonlinear stochastic systems with discrete-time measurements

Consider the problem of developing a controller for general (nonlinear and stochastic) systems where the equations governing the system are unknown. Using discrete-time measurement, this paper presents an approach for estimating a controller without building or assuming a model for the system. Such an approach has potential advantages in accommodating complex systems with possibly time-varying dynamics. The controller is constructed through use of a function approximator, such as a neural network or polynomial. This paper considers the use of the simultaneous perturbation stochastic approximation algorithm which requires only system measurements. A convergence result for stochastic approximation algorithms with time-varying objective functions and feedback is established. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite-difference gradient approximations.

[1]  J. Spall.  Implementation of the simultaneous perturbation algorithm for stochastic optimization , 1998 .

[2]  J. L. Maryak.  Some guidelines for using iterate averaging in stochastic approximation , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[3]  J. Spall.  Accelerated second-order stochastic optimization using only function measurements , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[4]  Yangsheng Xu,et al.  Human control strategy: abstraction, verification, and replication , 1997 .

[5]  Rui J. P. de Figueiredo,et al.  Learning rules for neuro-controller via simultaneous perturbation , 1997, IEEE Trans. Neural Networks.

[6]  J. Dippon,et al.  Weighted Means in Stochastic Approximation of Minima , 1997 .

[7]  D. C. Chin,et al.  Traffic-responsive signal timing for system-wide traffic control , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[8]  J. Spall,et al.  Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[9]  Payman Sadegh,et al.  Constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation , 1997, Autom..

[10]  M. Fu,et al.  Optimization of discrete event systems via simultaneous perturbation stochastic approximation , 1997 .

[11]  Richard D. Braatz,et al.  On the "Identification and control of dynamical systems using neural networks" , 1997, IEEE Trans. Neural Networks.

[12]  M. E. Ahmed,et al.  Neural-net-based direct self-tuning control of nonlinear plants , 1997 .

[13]  R H Smith,et al.  NETWORKWIDE APPROACH TO OPTIMAL SIGNAL TIMING FOR INTEGRATED TRANSIT VEHICLE AND TRAFFIC OPERATIONS , 1997 .

[14]  James C. Spall,et al.  A one-measurement form of simultaneous perturbation stochastic approximation , 1997, Autom..

[15]  Visakan Kadirkamanathan,et al.  Dynamic structure neural networks for stable adaptive control of nonlinear systems , 1996, IEEE Trans. Neural Networks.

[16]  F. Lewis,et al.  Discrete-time model reference adaptive control of nonlinear dynamical systems using neural networks , 1996 .

[17]  Edwin K. P. Chong,et al.  Analysis of stochastic approximation and related algorithms , 1996 .

[18]  K S Narendra,et al.  Control of nonlinear dynamical systems using neural networks. II. Observability, identification, and control , 1996, IEEE Trans. Neural Networks.

[19]  Hong Chen,et al.  Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems , 1995, IEEE Trans. Neural Networks.

[20]  H. Kushner,et al.  Stochastic approximation with averaging and feedback: rapidly convergent "on-line" algorithms , 1995, IEEE Trans. Autom. Control..

[21]  J. Spall,et al.  A neural network controller for systems with unmodeled dynamics with applications to wastewater treatment , 1994, Proceedings of 1994 9th IEEE International Symposium on Intelligent Control.

[22]  J. Spall,et al.  Nonlinear adaptive control using neural networks: estimation with a smoothed form of simultaneous perturbation gradient approximation , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[23]  Thomas Parisini,et al.  Neural approximations for multistage optimal control of nonlinear stochastic systems , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[24]  D. C. Chin,et al.  A more efficient global optimization algorithm based on Styblinski and Tang , 1994, Neural Networks.

[25]  Kumpati S. Narendra,et al.  Control of nonlinear dynamical systems using neural networks: controllability and stabilization , 1993, IEEE Trans. Neural Networks.

[26]  S. Yakowitz.  A globally convergent stochastic approximation , 1993 .

[27]  Dejan J. Sobajic,et al.  Neural-net computing and the intelligent control of systems , 1992 .

[28]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[29]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[30]  P.J. Antsaklis,et al.  Implementations of learning control systems using neural networks , 1992, IEEE Control Systems.

[31]  D.A. Handelman,et al.  Theory and development of higher-order CMAC neural networks , 1992, IEEE Control Systems.

[32]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[33]  Kurt Hornik,et al.  Convergence of learning algorithms with constant learning rates , 1991, IEEE Trans. Neural Networks.

[34]  Robert M. Sanner,et al.  Gaussian Networks for Direct Adaptive Control , 1991, 1991 American Control Conference.

[35]  Pierre Priouret,et al.  Adaptive Algorithms and Stochastic Approximations , 1990, Applications of Mathematics.

[36]  F. Girosi,et al.  Networks for approximation and learning , 1990, Proc. IEEE.

[37]  Arjan van der Schaft,et al.  Non-linear dynamical control systems , 1990 .

[38]  F.-C. Chen,et al.  Back-propagation neural networks for nonlinear self-tuning adaptive control , 1990, IEEE Control Systems Magazine.

[39]  D. Bayard.  A forward method for optimal stochastic nonlinear and adaptive control , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[40]  K Y San,et al.  The design of controllers for batch bioreactors , 1988, Biotechnology and bioengineering.

[41]  Engin Yaz,et al.  A control scheme for a class of discrete nonlinear stochastic systems , 1987 .

[42]  S. Evans,et al.  On the almost sure convergence of a general stochastic approximation procedure , 1986, Bulletin of the Australian Mathematical Society.

[43]  R. K. Miller,et al.  Stability Analysis of Hybrid Composite Dynamical Systems: Descriptions Involving Operators and Difference Equations , 1986, 1986 American Control Conference.

[44]  A. Michel,et al.  Stability analysis of hybrid composite dynamical systems: Descriptions involving operators and differential equations , 1985, 1985 24th IEEE Conference on Decision and Control.

[45]  D. Ruppert.  A Newton-Raphson Version of the Multivariate Robbins-Monro Procedure , 1985 .

[46]  Denis Dochain,et al.  Adaptive identification and control algorithms for nonlinear bacterial growth systems , 1984, Autom..

[47]  E. Eweda,et al.  Second-order convergence analysis of stochastic adaptive linear filtering , 1983 .

[48]  A. Benveniste,et al.  A measure of the tracking capability of recursive stochastic algorithms with constant gains , 1982 .

[49]  T. Soderstrom,et al.  Stationary performance of linear stochastic systems under single step optimal control , 1982 .

[50]  H. Anton,et al.  Functions of several variables , 2021, Thermal Physics of the Atmosphere.

[51]  H. Kushner,et al.  Asymptotic Properties of Stochastic Approximations with Constant Coefficients. , 1981 .

[52]  George N. Saridis,et al.  Self-organizing control of stochastic systems , 1977 .

[53]  George Tchobanoglous,et al.  Wastewater Engineering Treatment Disposal Reuse , 1972 .

[54]  D. Sworder,et al.  On the Control of Stochastic Systems , 1967 .

[55]  W. Rudin.  Principles of mathematical analysis , 1964 .

[56]  J. Blum.  Approximation Methods which Converge with Probability one , 1954 .