Global optimality of approximate dynamic programming and its use in non-convex function minimization

Level curves of the Rosenbrock function subject to minimization and state trajectories for different initial conditions x0ź{-2, -1, 0, 1, 2}×{-2, -1, 0, 1, 2}. The red plus signs denote the initial point of the respective trajectory. This study investigates the global optimality of approximate dynamic programming (ADP) based solutions using neural networks for optimal control problems with fixed final time. Issues including whether or not the cost function terms and the system dynamics need to be convex functions with respect to their respective inputs are discussed and sufficient conditions for global optimality of the result are derived. Next, a new idea is presented to use ADP with neural networks for optimization of non-convex smooth functions. It is shown that any initial guess leads to direct movement toward the proximity of the global optimum of the function. This behavior is in contrast with gradient based optimization methods in which the movement is guided by the shape of the local level curves. Illustrative examples are provided with single and multi-variable functions that demonstrate the potential of the proposed method.