An approximate dynamic programming based approach to dual adaptive control

Abstract In this paper, an approximate dynamic programming (ADP) based strategy is applied to the dual adaptive control problem. The ADP strategy provides a computationally amenable way to build a significantly improved policy by solving dynamic programming on only those points of the hyper-state space sampled during closed-loop Monte Carlo simulations performed under known suboptimal control policies. The potentials of the ADP approach for generating a significantly improved policy are illustrated on an ARX process with unknown/varying parameters.

[1]  Brian D. O. Anderson,et al.  Adaptive systems, lack of persistency of excitation and bursting phenomena , 1985, Autom..

[2]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Jay H. Lee,et al.  Approximate dynamic programming-based approaches for input-output data-driven control of nonlinear processes , 2005, Autom..

[5]  Warren B. Powell,et al.  Approximate Dynamic Programming - Solving the Curses of Dimensionality , 2007 .

[6]  Niket S. Kaisare,et al.  Simulation based strategy for nonlinear optimal control: application to a microbial cell reactor , 2003 .

[7]  K. Åström,et al.  Dual Control of an Integrator with Unknown Gain , 1986 .

[8]  Alan Jeffrey,et al.  Handbook of mathematical formulas and integrals , 1995 .

[9]  Jong Min Lee,et al.  Approximate Dynamic Programming Strategies and Their Applicability for Process Control: A Review and Future Directions , 2004 .

[10]  Jay H. Lee,et al.  Choice of approximator and design of penalty function for an approximate dynamic programming based control approach , 2006 .

[11]  Bengt Lindoff,et al.  Analysis of approximations of dual control , 1999 .

[12]  Takashi Yoneyama,et al.  A multistage suboptimal dual controller using optimal predictors , 1999, IEEE Trans. Autom. Control..

[13]  A. A. Feldbaum,et al.  DUAL CONTROL THEORY, IV , 1961 .

[14]  K. Åström,et al.  Problems of Identification and Control , 1971 .

[15]  Tamer Basar,et al.  Dual Control Theory , 2001 .

[16]  Y. Bar-Shalom,et al.  A multiple model adaptive dual control algorithm for stochastic systems with unknown parameters , 1979, 1979 18th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[17]  Stephen P. Boyd,et al.  Policies for simultaneous estimation and optimization , 1999, Proceedings of the 1999 American Control Conference (Cat. No. 99CH36251).

[18]  William R. Cluett,et al.  Stochastic iterative dynamic programming: a Monte Carlo approach to dual control , 2005, Autom..

[19]  Ahmed Ismail,et al.  Dual adaptive control of paper coating , 2003, IEEE Trans. Control. Syst. Technol..

[20]  Johan Wieslander,et al.  An approach to adaptive control using real time identification , 1971 .

[21]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[22]  N. Filatov,et al.  Survey of adaptive dual control methods , 2000 .

[23]  Björn Wittenmark,et al.  An Active Suboptimal Dual Controller for Systems with Stochastic Parameters , 1975 .