New designs for universal stability in classical adaptive control and reinforcement learning

Many researchers think that neurocontrollers should never be used in real-world applications until firm, unconditional stability theorems for them have been established. This paper explains key ideas from the author's previous paper (1998) which discusses the problem of "universal stability" (in the linear care) and proposes a new solution. New forms of real-time "reinforcement learning" or "approximate dynamic programming", developed for the nonlinear stochastic case, appear to permit this kind of universal stability. They also offer a hope of easier and more reliable convergence in off-line learning applications, such as those discussed in this paper or those required for nonlinear robust control. Challenges for future research are also discussed.