Non-convexities from probing in adaptive control problems☆

Abstract Computer codes for solving adaptive control problems frequently contain gradient methods. However, due to the presence of non-convexities in the cost function these codes may frequently obtain local rather than global optimum. This note uses the decomposition of the dynamic programming cost-to-go into deterministic, cautionary, and probing terms to show the source of the non-convexity in an example problem and to conjecture that such non-convexities may stem from fundamental properties of the probing term.