Finite-Horizon Discounted Optimal Control: Stability and Performance

Motivated by (approximate) dynamic programming and model predictive control problems, we analyse the stability of deterministic nonlinear discrete-time systems whose inputs minimize a discounted finite-horizon cost. We assume that the system satisfies stabilizability and detectability properties with respect to the stage cost. Then, a Lyapunov function for the closed-loop system is constructed and a uniform semiglobal stability property is ensured, where the adjustable parameters are both the discount factor and the horizon length, which corresponds to the number of iterations for dynamic programming algorithms like value iteration. Stronger stability properties such as global exponential stability are also provided by strengthening the initial assumptions. We give bounds on the discount factor and the horizon length under which stability holds and we provide conditions under which these are less conservative than the bounds of the literature for discounted infinite-horizon cost and undiscounted finite-horizon costs, respectively. In addition, we provide new relationships between the optimal value functions of the discounted, undiscounted, infinite-horizon and finite-horizon costs respectively, which are very different from those available in the approximate dynamic programming literature. These relationships rely on assumptions that are more likely to be satisfied in a control context. Finally, we investigate stability when only a near-optimal sequence of inputs for the discounted finite-horizon cost is available, covering approximate value iteration as a particular case.